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Foreword 


The National Forum on Education Statistics (Forum) is pleased to present the Forum Guide to 
Strategies for Education Data Collection and Reporting. The purpose of this resource is to provide 
timely and useful best practices for education agencies that are interested in designing and 
implementing a strategy for data collection and reporting, focusing on these as key elements of 
the larger data process. 


This publication builds upon Standards for Education Data Collection and Reporting (SEDCAR), 
available at https://nces.ed.gov/forum/pub 1991 92022.asp, which the Forum published in 1991 
to provide education agencies with best practices for the collection, processing, analysis, and 
reporting of education statistics. SEDCAR took a comprehensive view of the processes that occur 
during each phase of data collection and reporting, with six phases forming the conceptual 
framework: Management of Data Collection and Reporting; Design; Data Collection; Data 
Preparation and Processing; Data Analysis; and Reporting and Dissemination of Data. 


Because of the nature of data collection when SEDCAR was published, education agencies did 
not do nearly the level of compulsory and continual data collection that most do now. Moreover, 
large-scale data collections generally were conducted for research purposes, and few collections 
were coordinated as part of an agency’s overall data strategy. In contrast, this new resource is 
designed to be relevant to state and local education agencies (SEAs and LEAs) of today, in which 
data are collected regularly for multiple purposes, and data collection and recording may be 
conducted by many different individuals within an agency: teachers, administrators, analysts, 
or even students themselves. This new SEDCAR publication retains many of the relevant 
concepts and effective practices from the 1991 publication and expands upon the information 

to demonstrate how the concepts included in that document contribute to an effective data 
strategy. The new publication includes new and updated best practices, real-world examples, 
and timely resources that agencies can use to develop and implement a data strategy. 


Objectives 
This resource is intended to 


e address the needs of education agencies related to developing and implementing 
data strategies; 

¢ identify core practices for data collection and reporting; and 

e showcase different data strategy approaches used by SEAs and LEAs. 


Audience 


This resource is intended for staff and stakeholders in education agencies who are responsible 
for governing, collecting, and implementing data strategies and data collection and reporting 
practices. This includes federal, state, and local education agency staff who oversee data 
strategy development and implementation, as well as staff who are responsible for planning, 
coordinating, and implementing data collections and data reporting. 
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Organization of This Resource 
This resource is composed of three sections: 


1. An introduction to data strategy and its use in education agencies. 
2. An in-depth discussion of strategies for education data collection and reporting. 
3. Case studies from SEAs and LEAs. 


The information is presented in this manner to be as useful and responsive as possible to the 
needs of readers. Depending on an agency’s particular needs at a given time, one of the sections 
may be primarily relevant. Alternatively, many agencies will find all three sections applicable to 
their current needs. 


National Forum on Education Statistics 


The work of the National Forum on Education Statistics (Forum) is a key aspect of the National 
Cooperative Education Statistics System (Cooperative System). The Cooperative System was 
established to produce and maintain, with the cooperation of the states, comparable and 
uniform education information and data that are useful for policymaking at the federal, state, 
and local levels. To assist in meeting this goal, the National Center for Education Statistics 
(NCES) within the Institute of Education Sciences (IES)—a part of the U.S. Department of 
Education (ED)—established the Forum to improve the collection, reporting, and use of 
elementary and secondary education statistics. The Forum includes approximately 120 
representatives from state and local education agencies, the federal government, and other 
organizations with an interest in education data. The Forum deals with issues in education data 
policy, sponsors innovations in data collection and reporting, and provides technical assistance 
to improve state and local data systems. 


Development of Forum Products 


Members of the Forum establish working groups to develop guides in data-related areas of 
interest to federal, state, and local education agencies. They are assisted in this work by NCES, 
but the content comes from the collective experience of working group members who review all 
products iteratively throughout the development process. After the working group completes 
the content and reviews a document a final time, publications are subject to examination by 
members of the Forum standing committee that sponsors the project. Finally, Forum members 
review and formally vote to approve all documents before publication. NCES provides final 
review and approval before online publication. The information and opinions published in 
Forum products do not necessarily represent the policies or views of ED, IES, or NCES. Readers 
may modify, customize, or reproduce any or all parts of this document. 
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What is a Data Strategy? 


A comprehensive data strategy is a robust, integrated approach to using data to deliver on a 
mission, serve stakeholders, and steward resources while respecting privacy and confidentiality. 
A data strategy enables education agencies to leverage data to improve education, increase 
agency effectiveness, facilitate oversight, and promote transparency. Data strategies encompass 
data principles and practices such as governance, access, privacy, security, dissemination, and 
use by internal and external stakeholders. 


Data strategy can be considered the umbrella that encompasses all other aspects of the data 
universe within an agency, such as data collection, management, governance, analysis, or 
implementation. This allows agencies to carefully and deliberately consider the “big picture” 
of their data, thinking in terms of how and why they collect and store particular data, and how 
these data can be used most effectively to answer questions, solve problems, or plan for the 
future. Data strategy can be seen as integrating a system of people, policies, practices, and 
resources that are required to fully leverage the value of an agency’s data. 


The Need for Data Strategy 


A thoughtfully designed data strategy allows an agency to look beyond the day-to-day needs 

and purposes of data collection and use and think about the larger picture. A data strategy 
empowers an agency to think about how the information from different collections can be used 
to answer targeted questions, eliminate inconsistencies, clarify processes, and unify goals across 
the agency. Additionally, a clear, site-specific data strategy allows an agency to plan for the 
future, considering how current and future data provide the information needed to determine 
and work toward relevant goals. 


Many factors may influence how an agency approaches data strategy. Aligning the data strategy 
with the agency’s overarching strategic goals is a best practice. Particular aspects of an agency’s 
location, resources, or population served may influence the design or the approach to data 
strategy. Outside factors or influences, including unexpected changes or new dynamics, may 
drive the strategy. Ultimately, the path to and reasons for an agency’s data strategy are as 
individual as the agency itself. 


Data Strategies in Federal, State, and Local Education Agencies 


In the public sector, data strategy may include multiple government agencies working in 
coordination. Within the education community, interagency cooperation is more common in 
higher education and international education than in U.S. elementary and secondary education. 
When broadly implemented, data strategy in U.S. elementary and secondary education agencies 
may be a part of an interagency undertaking. When narrowly implemented, data strategy in an 
agency may be the responsibility of an individual team, division, or department. 
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At the federal level, the mission of the 
Federal Data Strategy (https://strategy.data. 
gov/) is to fully leverage the value of federal 
data for mission, service, and the public 
good by guiding the federal government in 
practicing ethical governance, conscious 
design, and a learning culture. The Federal 
Data Strategy offers guidance on how 
agencies should manage and use federal 
data, including principles and practices 

to deliver a more consistent approach 

to federal data stewardship, use, and 
access. While several of these practices are 
specific to the federal government, many 
of the practices may be useful to state and 
local education agencies (SEAs and LEAs) 
and can provide a starting point for data 


In 2018, the Vermont Agency of Education brought 
together data and measurement teams across the agency 
to create the Data Management and Analysis Division. The 
Division created the following set of strategic priorities: 


Modernize, standardize, and fully leverage collection, 
management, storage and data analysis platforms, 
tools, and methodologies. 

Move from a reactive culture to a proactive culture. 
Effectively coordinate to execute cross-functional 
workflows. 

Strengthen security and privacy frameworks while 
reducing burden of supporting secure and sound data 
handling. 

Empower the Vermont Agency of Education and 
stakeholders with data to support an evidence and 
result-based approach to decisionmaking. 


and administrative teams that are developing, revising, or expanding their data strategies. A 
complete list of the Federal Data Strategy Core Principles is provided in Appendix A. 


SEAs and LEAs are at different stages in their development and implementation of data 
strategies. Some may be just beginning to think of data from a big-picture perspective, moving 
beyond the immediate and practical use of data to design a strategy that will allow them to 

set short- and long-term goals for improvement. Others may be looking to revise or expand an 


existing data strategy. 


The Wisconsin Department of Public 
Instruction has a robust data system, called 
WISEdata, by which data from LEAs are 
collected in near real-time through an 
application programming interface (API). 
The data submission and any specific errors 
and warnings can be monitored by LEAs via 
a data quality portal. The SEA also reviews 
the data according to business rules needed 
for required reporting. The WISEdash data 
dashboard provides additional data quality 
metrics for both the SEA and LEA levels, 
including trend graphs to compare data over 
time. Overall, this process has improved 

the quality of data throughout the SEA and 
LEAs, and has streamlined and consolidated 
multiple data collections. Over time, this will 
decrease the effort needed at the LEA level 


Student Information Systems (SISs) 


SISs are secure information management systems 
designed to support all aspects of a student’s educational 
experience. They often include demographic data and 
information related to scheduling, attendance, discipline, 
health, grades, test results, and academic programs. 

They also may include data about economic status, 
accommodations, or geography. SISs allow local education 


agencies (LEAs) to communicate securely with teaching 
staff, parents, and students through web-based portals, 
modern phone apps, or traditional paper reporting. 


Definition provided by state education agencies (SEAS) 
and LEAs, including Fairfax County Public Schools (VA), 
https://www.fcps.edu/resources/technology/student-in- 
formation-system-sis-fcps 


since one system and process will be used for monitoring and maintaining data quality. 


The Delaware Department of Education considers its strategies for sharing data from the 
perspective of access channels, represented by a two-by-two matrix: internal/external on one 
axis and public/private on the other. Based on who requests the data and what type of data 
they request, the agency provides the data in a channel. It is working to streamline the channels 
through which internal and external users can access data. This will reduce the opportunities 
for consumers to receive multiple answers to questions and increase data privacy. 
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In recent years, the West Virginia Department of Education has expanded its focus on data 
quality, beginning with building ZoomWV-e, a single source data dashboard, which allows 
educators to see the data they enter into the state’s student information system (SIS) quickly 
and easily. The SEA also has developed an extensive data error report that is updated daily, so 
that local staff can see when data do not conform to expectations. The agency also holds regular 
internal meetings to discuss data quality issues. The agency’s data leaders believe that if data 
are not accurate, their usefulness is diminished—making all other data endeavors (reporting, 
funding, accountability) questionable. 


While data strategies differ among agencies, several core practices are useful for agencies to 
consider when establishing a data strategy:! 


1. Building a Culture that Values Data and Promotes Public Use 
2. Governing, Managing, and Protecting Data 
3. Promoting Appropriate and Ethical Data Use 


Building a Culture that Values Data Building a Culture of Quality Data 


and Promotes Public Use The Forum Guide to Building a Culture of Quality Data: 
The phrase “culture of quality data” is A School and District Resource was developed to help 
common to most who work with education schools and school districts improve the quality of data 
data, particularly at the LEA or SEA level. they collect. The resource discusses the importance of 
A culture of quality data refers to the helping all staff to understand how data will be used and 
idea that good data are an integral part of how data become information, and it shows how quality 
; : : BP data can be achieved through the collaborative efforts of 
teaching, learning, and managing schools, all staff. (https://nces.ed.gov/forum/pub 2005801.asp) 
and that everyone who has a role in student 
outcomes-—teachers, administrators, 
counselors, office support staff, school board members, and others—has a responsibility to 
promote high-quality data. Components of a culture of quality data include the following: 


e Accuracy. The information must be correct and complete. Data entry procedures must 
be reliable to ensure that a report will have the same information regardless of who 
completes it. 

e Security. The confidentiality of student and staff records must be ensured, and data 
must be protected from unauthorized access. 

¢ Utility. The data must provide the right information to answer the question that 
is asked.” 

e Timeliness. Deadlines are communicated, and data are collected promptly. 


At the school level, all staff must understand that data are entered into a data system so 

that information can be developed and used. One of the tasks of school administrators is to 
work with staff so that they can see how the information drawn from the data supports the 
school’s instructional program and business operations. Ensuring that those entering the data 
understand how the data are used, as well as their value, helps to emphasize the importance 
of their work and the quality of the data they enter. The goal is to establish conditions that 

will instill confidence and ownership for data quality among data users. Stakeholders who 

rely on reports must be able to trust that information is accurate, that the confidentiality of 
student records and the integrity of the data are maintained, and that they are getting the right 
information to answer their questions. 


1 These practices are drawn and adapted from the Federal Data Strategy: https://strategy.data.gov/. 
2 For more information on Data Use, see the Forum Guide to Taking Action With Education Data, available at https:// 
nces.ed.gov/forum/pub 2013801.asp. 
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LEAs also play a key role in ensuring Metro Nashville Public Schools (TN) takes a district-wide 


quality data. LEAs respond to the policies approach to data quality, with dedicated staff whose role 
and regulations set by state and federal is to identify data quality issues, discover root causes, and 
programs, as well as internal policies. work to resolve the issues at their sources to prevent them 
LEA personnel usually are responsible for from recurring. 


training data collectors and for ensuring 
that the data gathered are of high quality. 
They also may be responsible for dedicating 
resources to provide the best data collection 
environment possible for their schools. Data have incentives and/or penalties attached. 


The collection and use of education Data Baye tas tines. 
Districts see value in and use high-quality data. 


data have grown exponentially on recent Data are inspected for quality at the local level. 
decades, and as a result, education Data are inspected for quality at the state level. 
stakeholders better understand the 


critical need for quality data. Because of 
the increased understanding of and focus on high-quality data, this aspect of the data culture 
features significantly in many agencies’ data strategies. 


The Kentucky Department of Education emphasizes five 


elements of high-quality data: 


In many LEAs, creating a culture of quality data may be driven partially by the SIS vendor 

and the tools and procedures they have in place. SIS vendors implement requirements for 
collection and reporting within the SIS to help the LEA comply with state and federal reporting 
requirements. Training for the LEA staff responsible for the various required collections often is 
provided by the vendor directly as part of contracted services. 


One misconception that can occur when developing a culture of quality data within an 
organization is the assumption that once data enter a digital platform, such as an SIS, the data 
are clean and free of errors. As this approach to data quality develops within an organization, 
every database user must become a data curator to ensure high-quality data. Bozeman School 
District #7 (MT) has a developing data culture, with increasing responsibility for data quality 
across users. For example, the district has modified its procedures so that building secretaries 
review completed forms submitted by parents before approval and data entry into the SIS, to 
ensure accurate and complete student data enter the database. Gatekeeping in this way allows 
secretaries responsibility for the data for the students in their buildings. It also provides an 
opportunity for communication with families if data are missing or incomplete, before those 
data are needed during school operations. 


Milwaukee Public Schools (WI) has a key team that comes together with ancillary team members 
depending on the nature of the data. The district has instituted regular meetings to discuss data 
quality, better management of data quality, necessary professional learning, and procedures 

to maintain quality. The district also hosts monthly meetings with the SEA and other LEAs 

to discuss issues with the state and the SIS vendor. This has influenced how the district team 
looks at other collections, such as the Civil Rights Data Collection (CRDC) and key performance 
indicators (KPIs), as the team considers how to keep key data elements consistent across data 
collections. 
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Governing, Managing, and Protecting Data 

Data Strategy Models 

Depending on their needs and contexts, SEAs and LEAs may use any of the following data 

strategy models: 
¢ Single, agency-wide model. Many | Pata Strategy Checklist 


locations have an agency-wide A checklist can help ensure that an agency’s data strategy 
model, in which the strategy and addresses core concepts and key points. A checklist can 
related practices are specific to and__ | help an agency 

contained within the individual 


build in basics of good information practice; 


agency, and in which one strategy clearly define processes; 
guides the entire agency. work collaboratively; 

e Multiple-strategy model. Some build support/buy-in; _ 
agencies may have multiple baie ie who is responsible for each task within 
data strategies operating, with review the data strategy on a regular basis; 
departments or divisions designing have a schedule of who reviews the strategy and 
individual strategies to suit their how frequently; 


particular needs. consider the relationship between information 
¢ Interagency model. In an technology (IT) and data governance; 
: y del i 1 maintain a focus on cybersecurity; and 
interagency model, the plans, back up data. 
goals, and practices extend beyond 
a single agency to integrate with 
those of related agencies, for example, child welfare, health and human services, or 
higher education. 


Each of these models has benefits and challenges. For example, in a large LEA, different 
departments may be responsible for different data collections and quality assurance. The 
Milwaukee Public Schools (WI) Department of Research, Assessment, and Data is directly 
responsible for data that will be collected for state report cards and federal accountability 
reports. In particular, the department monitors dropout and graduation data. This requires 
relationships with other departments to maintain and report accurate data. For example, social 
workers reach out to families and collect data on dropout status, but if the secretary does not 
input the collected data with appropriate details and accuracy, data pulled from the SIS may be 
wrong or of poor quality. This results in multiple reviews of data. Additionally, the dropout data 
directly link to graduation data since the true enrollments determine the cohort size. 


The Delaware Department of Education uses a single, agency-wide model. Through the 
implementation of integrated data, the state is attempting to overlay an interagency model. 


The Wisconsin Department of Public Instruction, on the other hand, uses multiple models. 
The agency’s main strategy focuses on a single data standard, but it still has other legacy siloed 
collections and one-off collections that use a different method. Over time, these collections 
will be considered for integration into the single data standard. Additionally, the state’s Early 
Childhood Integrated Data System (ECIDS) has a third model that crosses multiple agencies. 
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Data Strategy and Data Governance 


Though data strategy and data governance are related concepts within an organization, they are 
not interchangeable or equivalent. 


e Data governance refers to a formal and comprehensive set of policies and practices 
designed to ensure the effective management of data within an organization. For 
example, an agency’s data governance practices would establish which individuals or 
offices within the agency are responsible for data elements, data sets, and databases. 
For further information, see the Forum Guide to Data Governance (https://nces.ed.gov/ 
forum/pub 2020083.asp). 

e Data strategy refers to a larger perspective that considers the agency’s goals and needs 
for the data, and how the data and their use fit into the organizational mission. For 
example, an agency’s data strategy might specify how the planned data collections align 
with the agency’s strategic plan. 


Therefore, an agency could have data governance practices in place for data collection and 
reporting, as well as a data strategy. The governance would clarify which data are collected, 
when collections occur, which individuals do which tasks, the means by which data are 
reported, and the expectations and requirements of the processes. The strategy would consider 
the reasons for collecting different data, the logic of having collections scheduled at particular 
times, the goals for reporting and use of the data, and many other aspects of how the data 
support the organizational mission. In short, an effective data strategy requires clear data 
governance. Data governance practices and policies also will be more relevant to stakeholders 
when they are seen as crucial to the larger data strategy. 


Promoting Appropriate and Ethical Data Use 


Each day, educators collect and use data about students, staff, and schools. Some of these data 
originate in individual student and staff records that are confidential or otherwise sensitive. 
Even the data that are a matter of public record, such as aggregate school enrollment, need to 
be accessed, presented, and used in an 
ethically responsible manner. While laws The National Forum on Education Statistics’ online 

set the legal parameters that govern data Data Ethics Course is based on The Forum Guide to Data 
use, ethics establish fundamental principles | E¢hics (attps://nces.ed.gow/forum/pub_2010801.asp) and 
of right and wrong that are critical to is focused on how ethical principles apply to education 


th int fand f data. The course is intended for any person who handles 
ay ped Pilate alee ae data in an education organization. https://nces.ed.gov/ 
education data. 


forum/dataethics course.asp 


The exponential growth of information 
systems that provide ready access to When agencies are not careful about ethical management 
education data—often drawing upon of data, situations can occur, such as embargoed data 
individual student records—has heightened being released or student lists with personally identifiable 
the importance of training data users about _ | information (PII) being emailed to others as a quick way 


their ethical responsibilities regarding to pass someone some data. These cases usually are the 
how they appropriately access, use, share, result of users not understanding the specific types of 
and manage education data. Technology data with which they are working. 

makes data readily available to many staff 


members in an education organization. To ensure proper dissemination, data should be marked 
While improved access helps staff perform (part of metadata) as to how they should be handled. 
their jobs more effectively, this access 
also raises issues about the appropriate use of data because the power to transmit information 
electronically multiplies the consequences of irresponsible behavior. 
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In the West Virginia Department of Education’s training for data tools, data leaders emphasize 
that the numbers in the reports represent real students: the children in schools that the state’s 
teachers and administrators try to care for and educate every day. It is important to help 
trainees focus on the students’ stories rather than just the numbers on the screen. 


A clear data strategy allows an education agency to design data collection and reporting policies 
and procedures that promote ethical data use. A data strategy also puts protections in place 

to identify and alert the agency to any potential ethical concerns. By incorporating the issue 

of ethics into the data strategy, an agency prominently indicates the value it places on ethical 
data use. Placing importance on ethics in the data strategy also ensures that the agency will be 
prepared with specific measures and responses to any unforeseen ethical breaches. 


Data use must be considered and designed carefully to ensure that it follows ethical standards. 
While it is necessary to collect some sensitive or personally identifiable data elements to ensure 
that students have access to services, access to these data and their use must be controlled 
carefully according to all federal, state, and local laws and regulations.* Beyond meeting 
regulatory requirements, anyone who works with potentially sensitive data should respect both 
staff and student privacy. For example, agencies should be aware that 


¢ Social Security numbers can indicate student immigration status;* 

e status flags for special education, homeless, National School Lunch Program (NSLP), 
pregnant minor, foster care, and other supports can detail confidential or private 
services that a student or family receives; and 

e third-party vendors providing online services (such as formative testing, remediation) 
often want various program and demographic flags to provide targeted reporting. 
Without proper data sharing agreements, collected student data and demographics could 
be used for research and marketing purposes without recourse by the LEA or SEA. 


Potential ethical breaches can occur at the school level if staff members are not properly 
educated. In some cases, the increased use of technology results in teachers discussing student 
needs via email, which can lead to careless sharing of sensitive information if all parties are not 
properly aware of protocols for protecting students’ personally identifiable data. It is critical for 
schools and districts to communicate the concept of collective responsibility and ensure that 
teachers and other staff members understand how to protect privacy. 


Similarly, staff members must be aware that 


using student data for any reason other than The Delaware Department of Education has handled 
the intended professional need is unethical. | the ethical issue of inconsistent redaction by employing 


While teachers and other staff members systematic redaction of data using software and 

may have access to the data platforms that publishing redacted data to the state’s open data 
provide student information, data use initiative. The state education agency (SEA) then directs 
other than use previously designated and over 95 percent of all external requests for public data 


approved is unethical and potentially illegal. | t© that data channel. This ensures that consumers get 
consistently redacted data, as opposed to asking the 


Data experts note that all interactions with same question at different times and getting data that 
student data have ethical ramifications. may not be redacted properly. 
For example, a social worker dealing 


3 For more information, see the U.S. Department of Education’s Privacy Technical Assistance Center (PTAC) 
resources, available at https://studentprivacy.ed.gov/. 
4 While Social Security Numbers (SSNs) may be needed internally to identify a student or to link together records 


from multiple sources, the SSNs should never be used in any type of data report. Once records are linked, they can be 
assigned a common ID apart from the SSN. If there is an expectation of the need to add data from another source in the 
future, the link between the SSN and the new ID should be maintained separately in a secure space. 
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with dropout data may have a direct conversation with a parent or student that allows them 
to document that the family has left the state. This student can be marked as “transferred out 
of state” legitimately. In another case, the social worker may have heard from friends of the 
student that the student moved out of state, but this secondhand information is not sufficient. 
This could result in the student being listed as a dropout, which is inaccurate, but ethical 
reporting demands that proper documentation be collected. 


Bozeman School District #7 (MT) navigates data ethics in a few ways. First, the district follows 
the strict Family Education Rights and Privacy Act (FERPA)-informed?® data security measures 
within the SIS, only allowing data access to users on a need-to-know basis. Security groups are 
set up in the SIS to ensure that sensitive student information is seen only by those who need 
access to it. The district also carefully considers what data are collected from families and how 
those data will be used. Questions about potentially sensitive issues like a student's academic 
history, family economic status, or race and ethnicity are requested only for required state 
and federal reporting or program participation (for example, the Individuals with Disabilities 
Education Act® [IDEA] or School Nutrition). Finally, the district has a comprehensive data 
governance document guiding the collection, storage, use, and disposal of student data. A staff 
data governance synopsis was developed from this document, providing staff with a quick 
resource on best practices around the ethical use of data and reinforcing the concept that 

the primary and fundamental use of student data collected by the district is to promote the 
academic and social-emotional success of students. 


5 For more information on the Family Educational Rights and Privacy Act (FERPA), see the U.S. Department of 
Education's FERPA resources, available at https://www2.ed.gov/policy/gen/guid/fpco/ferpa/ 
6 For more information on the Individuals with Disabilities Education Act (IDEA), see the U.S. Department of 


Education's resource, available at https://sites.ed.gov/idea/ 


8 Forum Guide to Strategies for Education Data Collection and Reporting 


State and local education agencies (SEAs and LEAs) regularly collect data for multiple purposes, 
and data collection and reporting may be conducted by many different individuals within an 
agency: teachers, administrators, analysts, or even students themselves. Data-related activities 
must be managed and coordinated to focus available resources where they are most needed, 
and in the most efficient and cost-effective manner. Additionally, processes must be put into 
place that provide the foundation for sound management and policy decisions about which 
data collection and reporting initiatives to pursue. Such decisions must be based on adequate 
information and must include the timely involvement and participation of stakeholders. 


Given the complexity of data collection and reporting, SEAs and LEAs increasingly are 
developing and adopting data strategies to clarify and maximize the purpose of their education 
data and effectively collect, manage, use, and protect those data. Section 1 of this resource 
provided an overview of data strategies in education agencies. This section discusses best 
practices for implementing one aspect of a data strategy—data collection and reporting. 


It can be helpful to consider the information lifecycle when planning a strategy for data 
collection and reporting. The information lifecycle is the series of steps needed in properly 
planning for, executing, and finalizing a data collection and the resulting uses and releases of 
data. The six phases of the information lifecycle: 


e Phase 1: Definition, Planning, and Development 
e Phase 2: Data Collection 

e Phase 3: Verification and Processing 

e Phase 4: Analysis and Use 

e Phase 5: Dissemination 

e Phase 6: Disposition 


Data do not exist in isolation. They are representations of the status of different parts of 

an agency and are interrelated and influential upon each other. Similarly, the clarity and 
effectiveness with which the data move through the phases of the lifecycle affect their quality 
and usefulness throughout the process. Because a data strategy allows an organization to 
consider how the collection and reporting of particular data can improve the organization’s 
functioning and allow it to reach specified goals, all parts of the lifecycle should be considered 
when planning and implementing that strategy. Across all phases, data need to be collected, 
managed, and used in ways that maintain their integrity, quality, and intended purpose. 


Forum Guide to Strategies for Education Data Collection and Reporting 9 


Phase 1: Definition, Planning, and Development 


Before data collection begins, agencies need not only to define the collection, but also to 
identify and justify collection activities, to ensure that the data are necessary, not redundant, 
and will be collected in a manner that allows effective use and interoperability. 


Checklist for Justifying Data ; 
Collection Activities Ensuring Data are Necessary 


1. Document the circumstances that Agencies should avoid collecting personally identifiable 
information (PID or other sensitive data unless they are 


absolutely necessary to the collection. For example, PII 


make the collection of information 


necessary, including any legal or likely is not necessary to a collection intended to assess 
administrative requirements. the school or grade as a whole. Collecting overlapping 

2. Indicate as specifically as possible information on different surveys or data collections can 
how, by whom, and for what increase opportunities for outsiders to crack security 
purpose the data will be used. measures on more sensitive items. 

3. Determine whether available data 
can be used to meet an emerging information need before initiating a new collection. 

4. Identify required data collection activities, as well as the accuracy and specificity 
necessary to achieve collection objectives. 

5. Analyze the costs and benefits of the proposed data collection to the producer and 
provider and, where appropriate, the costs of alternative strategies. 

6. Review the terminology and data definitions to be used in the data collection to 
ensure that they conform to accepted use. Any deviations from accepted use should 
be explained. Definitions should conform whenever possible to nationally developed 
definitions to ensure that the data produced will be comparable to data produced by 
education agencies and organizations at the school, district, state, and federal levels. 

7. Document data providers' concerns and data requestors' responses to those concerns. 


Managing for Interoperability 


In the past, separate data collections were Interoperability: the ability of different information 
managed independently, and this practice systems, devices, and applications (“systems”) to access, 


still is celuipiie saepe umes SEAS and LEAs. exchange, integrate, and cooperatively use data in a 
Many categories of data are required to coordinated manner, within and across organizational, 
be reported in various ways for various regional, and national boundaries, to provide timely and 
purposes, such as subgroups for state and seamless portability of information 

federal reporting. If the data are collected 
one way for one report and a different way 
for another, this can lead to complications for the agency. 


However, it is increasingly important that data are collected in a manner that meets as many 
needs as possible. For example, the same employee information may need to be collected by the 
human resources department and the information technology (IT) department, and collecting 
these data once and sharing them across departments increases both efficiency and accuracy. 
This practice helps to reduce burdens on agency staff by streamlining data collections (that is, 
they do not have to collect the same data twice), and it also improves data quality by ensuring 
that data stored in different systems are not contradictory. 


In many agencies, these processes still are not in perfect alignment. For example, in some 
states, districts may choose their student information system (SIS), which often differs from the 
state SIS. This puts the onus on districts to ensure that complete and accurate data entry occurs 
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in both systems. Without automated processes, data transfer must occur via manual entry or 
batch uploads. Manual entry into two systems can increase the opportunity for data entry error, 
and a focus on data quality in both systems becomes essential for accuracy in reporting and 
funding, and for facilitating student success. 


Designing the Data Collection 


Design is the process of formulating the primary purpose(s) for the collection and developing 
and describing a plan for conducting the collection, processing, analysis, and reporting of data. 
An effective design produces accurate and useful information, promotes timely and efficient 
data collection, and provides methods for resolving both expected and unexpected problems 
that may arise during data collection and analysis. 


Checklist for Formulating and Refining Data Collection Questions and Processes 


1. Ensure that the questions that drive the collection have the potential to address the 
data needs. 

2. Ensure that an individual question does not raise another question, or set of questions, 
that must be resolved before the current question can be answered. 

3. Ensure that the questions driving the collection do not make inaccurate assumptions. 

4. Ensure that questions do not pose an incorrect dichotomy (for example, make sure that 
the possible alternative answers are truly different). 

5. Ensure that questions do not attempt to resolve non-empirical problems by 
empirical means. 

6. Ensure that questions or collection processes have the same meaning for 
different persons. 


In addition to crafting appropriate data collection questions and processes, a key aspect of 
planning a data collection is to select appropriate sources of data. The following checklist 
provides considerations for those designing data collections. These questions may be relevant to 
arrangements made between LEAs and SEAs for the transfer of data, in which knowing what is 
feasible and realistic is critical to successful information sharing. 


Checklist for Selecting the Appropriate Sources of Data 


1. To identify feasible sources of information, ask the following questions: 
a. Can the information be obtained through analysis of existing data? 
b. Are records available from which the information can be compiled? 
c. Are subject matter experts or other knowledgeable persons available from 
whom the information can be gathered and assembled? 
d. Must the information be generated by controlled observation or measurement? 
2. To determine whether existing data can be used to answer the question, ask the 
following questions: 
a. Are data available that are relevant to the question? 
b. Do available data meet the criteria of reliability, validity, and other aspects of 
required technical quality? 
c. Are the data structured in a manner that provides the appropriate unit 
of analysis and that allows appropriate investigation of relationships 
among variables? 
d. Are the data sufficiently current? 
e. If multiple sources of data are used, are the data sufficiently comparable (for 
example, dates of collection)? 
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3. To determine whether administrative records can be used, ask the following questions: 

a. Are there administrative records that contain all the information needed (for 
example, numbers and characteristics of students by race/ethnicity and gender) 
for the administrative unit/level required to address the questions? 

b. Are there alternative methods of obtaining the records (once the administrative 
units in which the records are kept have been identified) that can accommodate 
different local recordkeeping, practices, and policies? 

c. Are the definitions and concepts employed by the various jurisdictions 
involved comparable and uniform? Be prepared to invest in methods (for 
example, crosswalking) that attempt to make data comparable among 
various jurisdictions. 

d. Does an examination of the uses for which the records are kept reveal clues 
about possible distortions relative to the questions? Administrative record data 
are no more immune to validity issues than data from any other source. 

4. To determine whether data can be collected from individual data providers, ask the 
following questions: 

a. Is there a person, position, or department with access to the information being 
sought? If there is, can that individual or department serve as the 
data provider? 

b. Is the data provider being asked to obtain information from administrative 
records? If so, consider the administrative record checklist above (item 3). 

c. Is the data provider being asked to report information about a group or 
organizational unit in the absence of records? 

d. Is the person from whom the data are being requested the most capable person 
to act as a data provider? If not, can a procedure be designed for choosing the 
best data provider? 

5. To determine whether data can be obtained via observation, ask the following questions: 

a. Is it feasible to train staff or to hire trained observers? 

b. Do standard protocols for observation exist, or can they be created? 

c. Can the observers be granted access to the phenomena of interest? 


If existing data are not available to answer the data collection questions, it may be necessary to 
design a data collection instrument. 


Checklist for Designing the Data Collection Instrument 


1. Provide clear and sufficient instructions for completing the data collection instrument 
(for example, survey, data submission tool, or report). Provide detailed instructions for 
individual items when necessary. 

2. Make definitions of data elements 
consistent with standard 


Common Education Data Standards (CEDS) 


definitions for those data The CEDS initiative is a national collaborative effort to 
elements, such as those provided develop voluntary, common data standards for a key set 
by the Common Education of education data elements to streamline the exchange, 

: comparison, and understanding of data within and across 
Standards (CEDS), when possible. P-20W institutions and sectors. For more information, see 


3. Provide definitions for any words 
in the data collection instrument 
whose meaning may be ambiguous. 


ceds.ed.gov. 
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4. Examine each item in the data collection instrument to make sure that the information 
is needed for data collection. 

5. Make sure that the purpose of each item on the instrument is understandable to the 
data provider. 

6. Explain to data providers why questions are included that have no apparent connection 
to the topic of the data collection. (For example, background questions might be asked 
to identify connections between people's backgrounds and their views on teacher 
competency testing.) 

7. Ensure that the requested information can be provided by the data providers. 

8. Minimize the amount of time data providers will need to complete the data 
collection form. 

9. Wherever possible, use units of measurement that are familiar to the data providers. 

10. Use standard language, and avoid jargon and abbreviations. Make sure that the 
technical terms used are appropriate to the data providers. Review questions for clarity. 
Keep questions short and simple. 

11. Design the item sequence of the data collection instrument to increase the data 
provider's ability to complete the data collection. Keep topic-related questions together 
and provide transitions between topics. Ensure that the item sequence does not 
influence responses to later questions. 

12. Make sure that the items on the data collection instrument place the least possible 
burden on the data providers. Find out how data providers usually keep or process the 
information being requested. 

13. Make sure that items do not combine two separate ideas inappropriately (for example, 
double-barreled questions) and that they ask for a single response. 


Phase 2: Data Collection 


Data collectors must handle myriad management activities effectively. Depending on the size of 
the collection, collectors may need to communicate with data providers at different locations 
and organizations, schedule data collection activities that may involve hundreds of people, 

and provide for the manual or electronic transfer of data from numerous collection sites across 
the district, state, or country. Several best practices improve data collection, especially when 
LEAs and SEAs may be handling large- - 
scale collections that cover many different The Kentucky Department of Education has developed 
areas of data. In short, the greater clarity a webpage with written guidance for local education 
provided to the data collection process, the agencies (LEAs) to help promote consistency in data 


h d fet lecti collections. The page includes information on the 
smoother and more accurate the collection importance of data standardization, data steward contact 


will be. Best practices include providing information for each standard, and links to individual 

written instructions, offering training and standards. The webpage is available at https://education. 
support, and establishing procedures that ky.gov/districts/tech/sis/Pages/KSIS-Data-Standards.aspx. 
increase accurdcy. 


Provide Written Instructions for Data Collectors 


It is important to provide coding instructions to data collectors in written form. This written 
guidance must be clearly worded, easily accessible, and customized to each data collector’s 
specific job. When written instructions do not exist or are subpar (that is, if they are poorly 
written, are not easily accessible, are not tailored to each audience, or are frequently or 
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haphazardly modified), data collectors might make coding assignments based on their 
understanding of the codes or other circumstances. Such independent interpretation of 
decision rules leads to inconsistent collection and decreases the data quality. 


Helpful instructional tools include not just explanations of what each code means, but 

also definitions of terms, flow charts for applying rules, and frequently asked questions. 
Customizing instructions for different audiences—such as school secretaries, guidance 
counselors, principals, and district technology staff—is especially helpful. Additionally, 
screenshots of how and where to enter the data can be helpful. Creating separate screenshots 
for every different potential data entry situation is not always feasible or desirable, but having 
basic, straightforward screenshots can be useful and help people feel more comfortable 
entering data, particularly if it is new to them. 


Provide Training and Support for Data Collectors 


Even when the data collection is well constructed and the collection instructions are well 
written, data collectors will need to learn to master the collection and coding protocols, and 
they may need ongoing monitoring and support. At times, well-trained, highly diligent data 
collectors still will face complicated — - - 
scenarios that require additional guidance The West Virginia Department of Education has a tiered 
from their agency’s data leads or outside system of support for questions about data collections. 
experts like data science professionals. West Virginia Education Information System (WVEIS) 


County Contacts are the first line of support; local staff 
Data collectors should not be forced to go to the County Contact first. If County Contacts do not 


wrestle with difficult choices in isolation; know the answer, the Contacts then reach out to the West 
instead, they should receive initial training | virginia Department of Education (WVDE), and the state 
and ongoing support for the duration of education agency (SEA) provides support (which County 
their jobs. Contacts then share back to their local colleagues). 


To ensure that data collection is being done 

appropriately and consistently, all data collectors should be provided with initial training and 
additional, periodic training as changes arise in the collection or instructional protocols. At the 
same time, they must have access to ongoing support—that is, they must have a place where they 
can reach out for assistance when problems arise. Many organizations offer a help desk service via 
telephone, email, or a website. Some organizations also maintain listservs that share out answers 
to all questions submitted by data collectors. 


Establish Procedures That Facilitate Accurate Data Entry 


Good procedures within education agencies can decrease data entry errors and thus improve 
data quality. For example, one best practice is to make staff assignments in such a way that 
routine reports are completed by the same people each cycle. For example, if a particular 
collection is completed three times per year, designate specific personnel to work on this task 
during all three time periods. Those personnel will develop expertise with that collection and 
thus make fewer mistakes. Unfortunately, frequent turnover or use of temporary staff can 
make this difficult in many agencies. However, when consistent connections between staff and 
collections are possible, this consistency can minimize problems. 


Another best practice is to manage the education agency’s office organization so that the 
designated data collectors have blocks of time during which they can enter data without 
unnecessary distractions. For example, school receptionists cannot be expected to enter data 
accurately if they are constantly being interrupted by other job responsibilities. 


Beyond these practices, agencies need to consider the specific ethical issues related to current 
technology, means of communication, and privacy laws. 
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Standard for Ethical Treatment of Data Providers 


Data collectors should ensure that the confidentiality of data is protected. Data teams should 
keep in mind that data providers are students, families, teachers, and other stakeholders, and 
that careful and thoughtful treatment of their data is critical. 


¢ Data collectors should not 
discuss confidential aspects of 


Forum Guide to Data Ethics 


the data collection activity with The Forum Guide to Data Ethics was developed in 
unauthorized individuals. response to the need among education organizations for 
* Copies of records, test scores, a simple, comprehensive set of standards for establishing 
and other data should be kept plans that encourage the ethical use and management 
: ; of data. It includes core principles (called “canons”), 
in a secure place and delivered examples, descriptions, and recommendations that 
promptly to the appropriate reflect real situations that arise in schools, school 
location or person. districts, and state education agencies (SEAs). 
¢ Notes and other documentation https://nces.ed.gov/forum/pub_2010801.asp 


kept during the data collection 
activity should not contain identifying information that is not expressly required by the 
research design. 

¢ Data collection activities should be carried out in compliance with applicable federal, 
state, and local laws concerning privacy and confidentiality. 

¢ Records should be destroyed upon completion of requirements for the data 
collection activity. 


Phase 3: Verification and Processing 


Planning for data processing should begin during the early stages of a data collection activity. 
Specific factors to examine when developing the plan include 


¢ types of data to be collected; 

¢ types of edit checks needed (such as verification of submissions of similar data 
across sources); 

¢ method for receipt control; 

* computer system to be used; and 

¢ timing and volume of data retrieval. 


Data verification is a process for checking the accuracy and quality of source data to ensure that 
the data are accurate and useful. There are three times during the process at which verification 
is crucial: point of entry, use, and application. At the point of entry, those entering the data 
should review the information carefully and take steps to ensure that all data are entered 

with precision. Before using or working with the data to answer questions or identify trends, 
individuals should perform data checks to verify their accuracy. These data checks should 
include consideration of thresholds and outliers to ensure that the data that have been entered 
are reasonable. Finally, data should be reviewed and verified once again before application, 

as a final check that they are worthy of being used as part of a solution, program, or another 
implementation effort. 
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Phase 4: Analysis and Use 


Data analysis is the process by which data are transformed into information that answers key 
questions and provides the foundation for decisionmaking. 


Reporting and Dissemination of Data 


Most data collection and analysis efforts culminate in one or more reports on data or the 
findings associated with them. Reporting can take many different forms, including formal 
written reports, data dashboards, or datasets that may be accessed and used for purposes of 
local, state, or federal reporting. For instance, the Elementary and Secondary Information 
System (EISi) (https://nces.ed.gov/ccd/elsi/) makes various types of reports available to 
stakeholders through quick facts and express tables while also enabling custom reporting 
through the table generator tool. 


Checklist for Planning the Report 
The plan for producing a report should include the following steps: 


1. Ensure that adequate resources are available for preparing and disseminating the report 

in the format(s) required. 

Ensure that realistic timeframes are set for producing the report. 

Identify intended audiences. 

Determine the audiences' information needs. 

Assess the audiences' level of technical knowledge. 

Identify the appropriate media for presenting findings to the intended audiences, such 

as online or printed copies. 

7. Follow appropriate protocols for dissemination to special groups (for example, board 
members or legislators). 

8. Consider if presentations are necessary to enhance dissemination efforts to particular 
audiences, such as boards, legislative education committees, or educational organizations. 


AN AWN 


Aligning Data Collection and Reporting Schedules 


Another best practice to consider when designing a plan for reporting is to align collection and 
reporting schedules. In many cases, effective use requires that data be timely. For example, 
data that depict the exit status of last year’s student population might be unsatisfactory for 
planners making programming decisions for the current school population. The most useful 
data, particularly at the LEA level, reflect the current state of the system and are available to 
decisionmakers when needed. Alternatively, state policymakers might need the most recent 
certified/finalized data for a completed school year. 


One best practice is to make sure that data collection schedules are timed for optimal data 
quality and use. Problems with data quality can arise when collections are scheduled too early 
in an academic year, before the information needed to accurately source the data is available; 
or too late in a year, when staff may be encumbered by other demands. Similarly, collections 
not timed to accommodate data demand (when stakeholders need the data) decrease the data’s 
utility and, subsequently, the quality of data-informed decisionmaking. 


A gap of time between collection and release is necessary so that personnel can verify the 
quality and organize the raw data into appropriate, usable formats. However, unnecessary 
delays might arise when reporting schedules conflict with holidays, releases of other types of 
data, or other times when staff are unable to dedicate time to releasing the data (or when data 
systems already are at working capacity). Unnecessary delays to data availability hamper good 
decisionmaking and should be avoided. 
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Changes in Compulsory Data Collection at the Local and State Level 


Over time, changes to compulsory data collection at the SEA, LEA, and school levels have 
affected data systems and overall data strategy in important ways. For example, changes in 
the past two decades in how variables such as race/ethnicity, as well as sex and gender, are 
categorized and reported have affected how data are collected and used. 


At the Vermont Agency of Education, changes in codesets (such as a change to provide gender- 
neutral or non-binary as codeset options) will necessitate changes in downstream reporting 
processes and products. There are costs related to these changes (for example, time, effort, 

or dollars to pay vendors for adjustments). Data leaders in Vermont note that these types of 
implementation costs always should be raised as part of the annual budgeting process. 


In Wisconsin, new state statutes for reporting additional information with state report cards 
have necessitated an increase of data collected in the current system. For example, under these 
statutes, Advanced Placement courses taken are reported on state report cards, which requires 
these data to be collected through student course roster data via WISEdata. A new level of 
quality checks was put in place to make certain the data transfer and data input are accurate. 


West Virginia’s Student DATA Act’ requires planning and public notification when the education 
department needs to collect new student data not already contained within the system. The 
timeline includes a 60-day public comment period with approval from the West Virginia Board 
of Education (both to release a proposal for comment and to accept the proposal after the 
comment period), making the actual timeline approximately 4 to 5 months for proposal and 
approval. State leaders therefore know to plan carefully and be intentional when proposing 

new data collections. Additionally, West Virginia’s data teams have found that local stakeholders 
need adequate time to adjust and to learn new tools, rules, or processes. When feasible, the data 
team tries to hold off on changes in the middle of a school year and implement changes for the 
following year. Because West Virginia has a statewide SIS created by and managed at the West 
Virginia Department of Education (WVDE), considerations about how to implement changes and 
the actual implementation of those changes are the purview of WVDE, with input from their local 
stakeholders. For example, if the WVDE receives approval to add a new data element at the start 
of a school year, WVDE staff add the ability for districts to collect that element into the SIS, but 
typically make that collection optional for the first year. Using this strategy, local users have the 
chance to learn the collection tool/item and definitions during the first year of implementation 
while the WVDE provides support and training about how to collect the data appropriately. 


Case in Point: California’s Changes in Data Collections Regarding Gender 


In 2016, the Federal Interagency Working Group on Measuring Sexual Orientation and Gender Identity (SOGI) 
released a paper describing how the concepts of SOGI are measured currently in U.S. Federal surveys. https:// 
nces.ed.gov/FCSM/pdf/buda5.pdf 


By August 2019, because of legislation changes, the California student data collection system (CALPADS) changed 
sex/gender data collection to allow students to declare themselves as non-binary. https:/Awww.cde.ca.gov/ds/sp/ 
cl/calpadsupdflash158.asp 


To be compliant with the changes, student information system (SIS) vendors have updated their systems so that 
they can report student gender within the guidance provided by CALPADS. These changes trickle down to the local 
education agency (LEA) level with procedural changes being implemented by administration and put into effect by 
registrar staff charged with collecting this information. 


7 For more on West Virginia's Student DATA Act, see http://www.wvlegislature.gov/WVCODE/ChapterEntire. 
cfm?chap=18&art=2&section=5H#02 
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Data Privacy and Security 


Rapid advances in technology have changed 
the nature of data collection and storage. 
They also significantly influence how LEAs The Forum Guide to Education Data Privacy is a 

and SEAs look at data privacy and security, resource for state and local education agencies (SEAs 
both daily and as a critical part of their data | and LEAs) to use in assisting school staff in protecting 
strategy. Agencies must consider issues such | the confidentiality of student data in instructional and 
as cloud storage, third party agreements for | administrative practices. The resource provides an 
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storing data, and how ownership of data overview of education data privacy, including relevant 

influences privacy and security. laws, and case studies in protecting student privacy in 
SEAs and LEAs. 

In Milwaukee Public Schools (WI), the https://nces.ed.gov/forum/pub_ 2016096.asp 


agency has used security management 
that comes with data systems. The agency 
developed roles and access by role within the system, ensuring that only individuals who need 
to see the data have access. The agency has a team that reviews requests to change roles and 
determines whether the need and purpose are legitimate. 


Vendors, grant agencies, and researchers now request data more than ever from schools and 
districts. All members of an organization should be aware of processes such as data agreements 
or institutional review board regulations to ensure that approval has been granted before 
releasing data. Organizations must have appropriate representatives review data requests to 
ensure that the requests benefit the organization and that data security will be maintained by 
the requesting agency.® 


To help education agencies, school officials, teachers, parents, and other education stakeholders 
understand and implement the requirements of the Family Education Rights and Privacy Act 
(FERPA), the U.S. Department of Education established the Privacy Technical Assistance Center 
(PTAC) within the Student Privacy Policy Office. PTAC offers a variety of resources related to 
student data and student data systems, including publications, training materials, and technical 
assistance. Resource topics include data privacy, confidentiality, and security practices.° 


Maintaining Best Practices in Data Management 


As times change, traditional ways of looking at data collection and management may no longer be 
timely or complete. Things that were state-of-the-art or best practices in previous years may no 
longer be enough to handle current needs, data uses, or technologies. Awareness of these shifts in 
best practices, and regular review of practices and standards, is crucial to effective data strategy. 


In recent decades, the way that education agencies look at many issues has shifted necessarily. 
These issues include 


e regular evaluations of ongoing data collections; 

¢ standard definitions of data elements; 

e determining costs and benefits of data collection; 
e justification for data collections; 

e data destruction; and 

e data sharing agreements. 


8 For more information on developing agreements to share data with researchers and others, see the Forum Guides 
to Supporting Data Access for Researchers: A Local Education Agency Perspective and A State Education Agency Perspective, 
which are available at https://nces.ed.gov/forum/pub_2014801.asp and https://nces.ed.gov/forum/pub_2012809.asp. 

9 For more information about PTAC, see https://studentprivacy.ed.gov/. 
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Phase 5: Dissemination 


Data dissemination is the sharing, o. ; 
distribution, or transmitting of data to When the West Virginia Department of Education 
stakehold es users. or other interested transitioned to the ZoomWV dashboards for public 
parties. Da ta may ba shared in multiple reporting, staff heard from a lot of people who liked the 


f hiaed fil : old format for certain reports. In response, the state 
ormats, such as data Iles, summaries, or education agency (SEA) started producing extra reports 


Working with Your Audience 


electronic or paper publications. (in spreadsheet format) so that people could download the 
Checklist for Presenting Data and reports in the formats they liked from the dashboards. 
Findings in a Manner that is Appropriate 
for the Intended Audiences The agency also tracks data requests to see if 
: ; multiple similar requests from different stakeholders 
1. Consider producing separate might indicate a need for a new dashboard or new 
reports for selected audiences. supplemental downloadable report. 


2. Make reports prepared for the 
public easy to understand, as well 
as technically accurate. 

3. When a narrative is needed, write Effective data visualization is an important part of 
in straightforward, nontechnical Saas as in that il hu easter te : 

Mak mor T-ir1len nN K 
language foune degree Takine xe ‘i ne The Forum Gude to Data Visualeation a 
subject matter permits. . designed to help education agencies communicate data 

4. Jargon, regional terms, and the like | meaning in visual formats that are accessible, accurate, 
should be avoided. and actionable for a wide range of education stakeholders. 

5. If diverse audiences are expected https://nces.ed.gov/forum/pub 2017016.asp 
to read a report, use subheadings 
and summary data to assist the 
various audiences in locating salient information. Consider presenting reports or 
data in multiple languages based on the most prevalent language groups in the area. 
Additionally, consider the range of potential web access by different individuals, and 
adhere to web accessibility standards. 

6. Consider using a variety of methods for communicating information about the data 
collection activities. Brochures, fact sheets, videos, and slides may be used in addition 
to or in place of traditional narrative reports. 

7. Ensure that all Americans with Disabilities Act of 1990 (ADA) requirements are met in 
public reporting products. 

8. Include the data source and date data were pulled. 
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The Key Role of Metadata 


Metadata are data that classify or otherwise describe other data. Metadata are crucial to data management 
because they clarify how to search for particular data and understand how they were collected. They also 
provide information about methods, participating agencies, data sources, and temporal and spatial markers that 


may be relevant to future data users or researchers. 


For more information, see the Forum Guide to Metadata, available at https://nces.ed.gov/forum/pub 2009805.asp. 
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Phase 6: Disposition 


The final stage of the information lifecycle, 
data disposition, is the destruction of the 
data or transfer of the data to an archive 
for future reference. In short, disposition 
refers to the decisions made about the 
future usefulness or need for the data, 
consideration of security issues raised 

by keeping or destroying the data, and 
plans for its final condition. Though a 
large amount of potentially sensitive 
student data may become unnecessary 

or irrelevant when a student graduates 

or otherwise leaves the school,!° some 
parts of the records of former students are 
required for varied purposes. For example, 


The Importance of a Clear Disposition Plan 


In West Virginia, some districts are digitizing old 
records and purging the physical files to help control 
storage space issues. Some of the data experts involved 
worry about destroying the “permanent records” of 
former students, and these concerns may be valid. For 
example, the education record might be the only record 
of vaccinations a person can get, or people may need 
education records to validate their age or eligibility for 
programs like Medicare and Social Security. Having a 
plan to digitize the relevant information can help the 
district maintain the information it thinks it needs to 
keep to be ready to assist former students, while still 
destroying (in an appropriate manner) old files or 
information that no longer is needed. 


students’ transcript information may need to be preserved indefinitely, whereas other student 
information will need to be preserved for a prescribed time period to comply with legal or 
policy requirements governing record retention. Agencies also should consider accessibility 
by various types of staff (for example, compliance officers or staff responsible for sending 


transcripts) in the retention strategy. 


One of the last decisions in the lifecycle of information comes when specific data cease to be 
accessed and used for the purposes for which they originally were collected and stored. When 
they are dormant yet still occupying valuable storage space, a decision must be made whether 
to archive or destroy the data. Some data by their nature are eternal, and must be properly 
and securely archived in case they are ever needed again (for example, transcript and financial 
data). Other records eventually will lose their value and should be destroyed in a manner 
consistent with their sensitivity. It is a best practice for data teams to review any relevant 
retention requirements for state and federal programs as they determine disposition plans. 


Best Practices for Data Destruction 


The U.S. Department of Education’s Privacy Technical Assistance Center (PTAC) published Best Practices for 
Data Destruction to provide agencies with best practices for properly destroying sensitive data after the data are 
no longer needed. The resource details the lifecycle of data, discusses various legal requirements relating to 


the destruction of data under the Family Educational Rights and Privacy Act (FERPA), and examines a variety 
of methods for properly destroying data. It also discusses best practices for data destruction and provides 
real-world examples of how to implement data destruction in an education agency. For more information, see 


https://studentprivacy.ed.gov/resources/best-practices-data-destruction. 


10 Privacy Technical Assistance Center (PTAC). (2014). Best Practices for Data Destruction. https://studentprivacy. 


ed.gov/resources/best-practices-data-destruction. 
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This chapter provides case studies from state and local education agencies (SEAs and LEAs) that 
discuss the specifics of their agencies’ data strategy plans, how their overall data strategy was 
envisioned and developed, and potential challenges and solutions they experienced along the 
way. These case studies also include SEAs or LEAs that still are within the development process 
for their data strategy, as the details and nuances of creation and implementation are likely to 
be useful to readers. 


Washington State Office of Superintendent of Public Instruction: The Case for Data 
Strategy Documentation 


The Washington State Office of Superintendent of Public 


Though many SEAs and LEAs have well- Instruction’s “why” for data strategy is that the data 
considered data strategies, they often are only as good as the information behind them, 
have varying levels of documentation of and what that information represents. Policy decisions 
their overall strategy or specific processes. require accurate and relevant data. 


Agencies that have clear documentation 
are in a much better position to navigate through changes or transitions. For example, during 

a recent change in administration, the Washington State Office of Superintendent of Public 
Instruction" found how advantageous its existing documentation of the agency’s data strategy 
was for the numerous stakeholders involved in the transition. In the data governance manual,” 
which was published publicly and created using significant stakeholder input, the data 
governance team had not only made a clear case for the “why” of the overall data strategy, but 
also why particular processes were in place and the origin of those processes. This meant that 
the incoming new administration found a concise, clear document waiting for it that provided 
rapid comprehension of the agency’s existing data processes, as well as the carefully considered 
reasons for them. 


With this documentation in place, the new Superintendent of Public Instruction quickly 
understood the agency’s intentions for different data, how different offices functioned and 
worked together in data collection and reporting, and the specific roles of individual staff 
members and teams. The new administration also made a point of meeting with people across a 
range of positions and departments, asking them to describe their roles and how they fit into the 
larger system and contribute to the agency’s mission. This approach to the transition allowed 
the new administration to understand the history and goals of the agency’s data strategy, and let 
it expand beyond the foundation of the strategy documentation. 


il For more information on the Washington State Office of Superintendent of Public Instruction's K-12 Data 
Governance workgroup, see https://www.k12.wa.us/about-ospi/workgroups-committees/currently-meeting-workgroups/k- 
12-data-governance 

12 Washington State Office of Superintendent of Public Instruction. (2015). Data Governance System for K-12 Data: 
Policies and Procedures. https://www.k12.wa.us/sites/default/files/public/cis]/pubdocs/DataGovernanceManual.pdf 
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This view of a data strategy as part of a 
larger dynamic system is a departure for 
the agency, which previously had focused 
on data for compliance with state and A strong data strategy considers all levels of stakeholders 
federal reporting requirements. A clear data | and includes all parties in strategizing. When you 
strategy now is a key part of the culture. As _ | havea good structure in place to ensure that you get 


Key Insight from the Washington State Office of 
Superintendent of Public Instruction 


new team members come in, there is no information from all levels, people can ask questions, 
need for major changes or fixing processes, _| Offer insight, and provide clarity. Supporting and 
because what is in place is defined clearly documenting all those voices makes the information that 


you can provide meaningful at all levels. 


and working for relevant parties. 


The state’s clear data strategy also helps 

the agency to meet the state superintendent’s policy and leadership expectations. For example, 
when the state superintendent and the state legislature were interested in expanding and 
providing funding to districts for dual language programs, the agency needed to address the 
issue that no federal data reporting about these programs existed and therefore no quality 
information was readily available. The agency worked with its steering committee to determine 
what data would need to be collected, which led to new conversations with schools and 
districts about the reporting process. It established a data collection to investigate where such 
programs exist, how many students are served, how many teachers are involved, what types of 
certifications these teachers have, and which parts of programs are going well or need further 
strategies to improve. This allowed the development of a timeline to expand the language 
immersion programs using a phased-in approach. This timeline gave the superintendent 

the opportunity to monitor progress toward goals, while also giving schools and districts 
opportunities to learn what other locations were doing and why different data were collected. 
In short, they were able to strategically consider various needs and uses for the data, including 
how they might be used beyond the immediate request. This perspective on data strategy 
allowed a more thoughtful discussion and plan for the dual language data. 


Challenges: Coaching Data Leaders and Aligning Processes 


While its perspective on data strategy has offered Washington many advantages, the agency 
has nonetheless faced some challenges. For example, as new cabinet members have come 

in from other parts of the agency or other locations, they have needed to be educated about 
the agency’s systems, particularly about the collection, reporting, and use of data. These 

new cabinet members may have had varying levels of familiarity with or use of data in their 
prior positions and now need to understand and use data with confidence to lead their teams 
successfully and have their voices heard. The state’s data governance leaders have focused on 
effective coaching of individuals in their new roles, as well as collaborating with related teams, 
such as information technology (IT), who can support the same messages. 


Data leaders also acknowledge another data strategy priority—the continued alignment of the 
processes of different teams and content areas to make them more consistent. A history of more 
siloed data activities meant that groups developed different processes as they were needed over 
time, but the current need for interoperability and data sharing requires a streamlined process 
and consistent understanding of data policies. The SEA is continuing to work toward greater 
alignment. This also will benefit the IT group, which is working to move away from its history 
of building custom systems to meet the needs of different groups, and instead, identify flexible 
technology tools that can be used for many purposes. With the data governance and IT teams 
working together to align data collection and reporting processes to meet the state’s larger data 
strategy, the SEA intends to not only improve the quality of the state’s data but to focus on long- 
term goals for the state’s technology. 
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Pasco County Schools (FL): Using a Data Challenge to Improve District Coordination 


Pasco County Schools (FL), in Land O’ Lakes, Florida, has turned a data challenge into an 
opportunity for greater clarity and improved data sharing among districts. Initially, the district 
experienced a complication—when data were intended for a specific purpose across different 
district information systems, the meaning of these data was not made adequately clear. 
However, upon becoming aware of the issue, data officials worked with colleagues in other 
districts, as well as the Florida Association for Testing Administrators (FATA; https://www. 
floridatestadmin.com/), to correct the confusion and use the situation to guide future decisions. 


The Challenge: Non-Reportable SAT and ACT Scores 


To meet graduation requirements for the state, students in Florida are allowed to take the 

SAT or ACT exam under a “non-college reportable” status—meaning that these scores are not 
intended to be reported to colleges or used for purposes other than high school completion. 
Students have the option to take these SAT NCR or ACT NCR (NCR meaning non-college 
reportable) exams in addition to the required state end-of-course (EOC) tests in English 
Language Arts (ELA) or Algebra if they have not scored at least at the required Level 3 on these 
EOCs. Students are entitled to extra time when taking NCR versions of the SAT or ACT, without 
the requirement of an Individualized Education Program (IEP). With this adjustment, many 
more students achieved scores high enough to qualify for graduation. 


Though this allowance benefitted many students in terms of graduation requirements, Pasco 
County Schools (FL) confronted a dilemma, in that the student information system (SIS) did 
not mark these scores as non-college reportable. This created both internal and external 
problems. Internally, it was sometimes unclear whether a score should be included when SAT 
or ACT scores were used for reporting. Externally, students would encounter problems when 
transferring to other districts. These scores traveled with them, and the receiving districts did 
not recognize their non-reportable status. 


Solving the Problem 


Because there was no state code for 
these data at the time, the lack of clarity 


The Florida Association of Test Administrators (FATA) 
(https://www.floridatestadmin.com/) is a grassroots 
organization that brings district test coordinators 


about these scores caused a great deal together to share best practices and address concerns. 
of confusion, particularly in the cases of The group is divided into regions and holds regional 
transferring students. Luckily, the problem | meetings and an annual state meeting. FATA allows 

was discovered quickly in Pasco County. members to problem solve any issues dealing with 

By the following test administration, assessments and also provides an opportunity for those 
the county had set up subject codes from similarly sized districts to share ideas and solutions. 


for non-college-reportable scores and 
added an “NR” flag. At this point, when IT staff load the test scores, they add a clarification of 
“transcript=NO” so that the score cannot leave the district on a transcript. The information 
based on these scores is reported only as “graduation requirement met." 


In addition to other districts making similar adjustments to avoid this confusion, FATA members 
have discussed these concerns at their meetings, both to mitigate the immediate problem and to 
consider how this situation can be a learning experience for future data sharing. 


Moving Forward 

Data leaders in Pasco County Schools (FL) acknowledge that situations such as this one cannot 
always be foreseen or prevented. Instead, rapid recognition of the issue and quick mitigation 
allowed the county to keep data confusion from becoming an ongoing problem. This situation 
did have positive consequences, in that it provided district leaders an opportunity to reflect 
on questions that need to be asked when adding new data metrics to the system and to think 
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about potential unintended consequences or impacts. Additionally, having worked through 
this problem, they are better able to anticipate and avoid similar problems in the future. They 
aim for a collaborative focus. For example, the Director of Accountability, Research, and 
Measurement works closely with the IT director to coordinate issues such as data sharing and 
data visualization tools. Similarly, instructional staff members meet with school support staff 
and the data visualization team, so that the different groups better understand each other’s 
needs and perspectives. 


Vermont Agency of Education: Redefining Structures for Data Strategy 


The Vermont Agency of Education’s data strategy development is an example of how a state can 
rapidly change its approach to data strategy and reorganize to support its new objectives. Until 
recent years, the state did not have an official data strategy. Some departments had strategies 

at varying levels, but these were relatively lean and did not transfer from one group to another. 
Following the entrance of a new education secretary, Vermont’s approach to data strategy has 
been reworked and redesigned, with a focus on bringing together all mission-critical data teams 
to work together. A new Data Management and Analysis division was created, aligning what 
previously had been a loose federation of data teams in different content areas. The state has 
made strides in recent years and continues to strengthen and implement its data strategy. 


Within this new division, the director created a data leadership team with all team leads. 

This group has created a mission statement and spent significant time identifying issues it felt 
would be game changers for the agency to develop a comprehensive charter. The team has 
had multiple strategic planning sessions, investing time in developing a clear data strategy and 
documenting all details. 


Going forward, each data leader will be responsible for one of the annual objectives. Leaders 
were able to choose their objectives, which increases the level of ownership. The team is 

in the process of revisiting what the key performance indicators (KPIs) are for each annual 
objective, and how they will be measured. For each objective, the team identified strategies 
and specific tasks. 


Key Elements of the Vermont Agency of Education’s Data Strategy 


A central element of the Vermont Agency of Education’s data strategy is its focus on 
standardization. As the state works to rapidly improve its systems from a technical perspective, 
staff members are dedicated to documenting everything that occurs within the shared space. The 
data division has a workflow/shared project management system with the IT department, with 
tools such as a Kanban board (a project management tool designed to visualize work processes 
and workflows) to show task dependencies. As the team operationalizes different pieces, it always 
begins with the business process map, considering what the workflow needs to look like, how to 
assign tasks and set estimates for time duration and deadlines, and how to best build the collective 
knowledge base. With these processes, it seeks to build a culture of agile work. 


Additionally, the team tries to be methodical about modernization and is focused on making 
sure things are no longer siloed within different groups. Team members created standards for 
workflows that allow them to move away from individuals owning processes to more of the 
team having ownership. The team uses a collaborative notebook software tool that allows team 
members to work within a data science environment. Each team member has a professional 
development plan geared toward modernizing skill sets, as well as understanding the best tools 
and means for executing their tasks. 


Related to its efforts to modernize its systems to better allow team members to work 
collaboratively is the data division’s emphasis on working from a data lifecycle perspective. 
The data division has moved away from seeing tasks as specific to separate teams, and instead, 
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focuses on the lifecycle, such that everyone has an understanding of both upstream and 
downstream work. It has eliminated the idea of sequestered spaces within a project, aiming 
instead to have everyone understand how their work affects the other parts of the process. The 
division has found that it is good for team morale when staff members see how their discrete 
tasks contribute to the larger project. This perspective also reemphasizes the importance of 
data quality. 


Finally, a central element of Vermont’s efforts to move data strategy forward has been the close 
relationship between the data division and IT. The director of the data division collaborates 
closely with the head of the IT department, allowing planning and decisions to be collaborative. 
The two groups have conducted an exercise to determine the roles and responsibilities of each 
department. In short, who should be doing what? Where does data stop and IT start, and vice 
versa? Though these are sometimes tough conversations, they have allowed a much greater 
understanding of roles within the data process and have provided clarity of purpose. 


Moving Forward 


As Vermont moves beyond various data strategy goals tied to modernization and 
standardization, the state also is focusing on reducing state reporting burdens. Taking concerns 
from the districts into consideration, such as their frustration that they are unable to help each 
other or to share data, the state recently released a request for information (RFI) for a statewide 
SIS. Although Vermont is a local control state, it is exploring the statewide SIS route due to the 
extreme burdens LEAs report in simply meeting compliance requirements. 


If a statewide SIS were to be implemented, the data division hopes that within 3 to 5 years, 
state reporting burdens (particularly time burdens) could be reduced by up to 80%, as the 
state could draw much of the needed data from the SIS. It also would enable the state to begin 
cycling through data quality checks throughout the year so they would not be an enormous 
undertaking a few times a year. 


With necessary data reporting efforts reduced, Vermont could turn its focus to using the data to 
do meaningful things in the field. The data division plans to use the data for strategic purposes 
such as program evaluation, early warning systems, and process improvement. 


West Virginia Department of Education: The Importance of Being Intentional 


In the process of moving from an administrative or compliance-based data focus to one that is 
more strategic and future-focused, the West Virginia Department of Education has centered on 
being intentional, such that those who work with data are effectively supported, dissemination 
decisions are made judiciously, and data are recognized as representations of real people. 
Ultimately, state leaders want to ensure that data are used as strategically as possible to drive 
the decisions that support the state’s children. 


Support and Training for Data Collectors 


This intentionality begins with the state’s focus on support and training for data collectors 

at all levels. Until recently, West Virginia had a tiered system of support that included a layer 
of regional education service agencies (RESAs) between the SEA and LEA levels. Under this 
system, data collectors at the school level directed questions to designated representatives 

at the LEA, and the LEA would contact the RESA. If the RESA could not answer, RESA 
representatives then went to the SEA. In the wake of changes in state law that eliminated 
RESAs, the state has streamlined this process while still keeping a tiered system. Currently, 
questions from the school level go to the WVEIS (West Virginia Education Information System) 
County Contact within each LEA. These individuals can contact the SEA when needed, as in 
the past. However, because the RESAs have been eliminated, the SEA meets more regularly 


Forum Guide to Strategies for Education Data Collection and Reporting 25 


with the WVEIS County Contacts (for example, via virtual meetings and in-person conferences) 
to keep them updated on important issues. With increased collaboration and improved 
communication from the SEA, the WVEIS County Contacts are better able to build the capacity 
to handle local questions and situations directly. 


Though WVEIS has existed for several decades, the state has worked in recent years to 

more intentionally establish coding standards. Traditionally, LEAs had divergent methods of 
defining and collecting data elements. For example, for many years LEAs defined the concept 
of “absence” differently—in some LEAs, a student absent due to a chronic medical condition 
might be counted as absent because they are not physically in class, while in other counties, 
the same type of absence might be defined as an “allowable deduction” that would not count as 
an absence. By standardizing how such concepts could be defined (through changes in policy 
and practice), the state was able to make the data more consistent and accurate. Also, the SEA 
framed the issue for LEAs as one of fairness—for all to be treated equally, data definitions and 
collections need to be consistent. 


The focus on intentionality also extends to changing processes or collections. In one prior case, 
the SEA expected data collectors (school secretaries, in this case) to learn a new process quickly 
before the start of a new school year. The SEA’s expectation was unreasonable because it did 
not allow requisite time for school secretaries to adjust and learn what was needed. Therefore, 
the process change had to be eliminated. The state learned from the experience and now uses 
phased-in plans to allow for effective transitions. 


Finally, the West Virginia Department of Education also carefully considers the level of 
instruction provided to various data collectors. The SEA provides detailed data collection 
instruction documents on how to use SEA applications, which include an interactive table of 
contents that users can click through to find the information they need. Detailed instructions 
for major data collections include contact information for particular content areas, specific 
instructions, key codes, and screenshots of individual steps. State leaders acknowledge that 
such a comprehensive document can be daunting, but explain that a decision was made to 
provide all needed information in one place rather than attempting to maintain several separate 
documents for different portions of the same collection. WVEIS County Contacts are available to 
answer questions and provide support. 


Dissemination 


The West Virginia Department of Education also has considered its various stakeholders in its 
data dissemination strategy. Some of this has been dependent on the type of data requested, 
and some has been in response to stakeholder reactions to changes in dissemination. 


For standard dissemination, the state has a public dashboard reporting site, as well as a state 
report card site. They ensure that there are various downloads available, noting that different 
stakeholders have different interests in and uses for the public data. State data leaders note 
that when these options were first available, some users did not want to download the data 
for each county or school individually, as allowed for by standard dashboard functionality: 
They wanted the spreadsheets to which they were accustomed. In response, the state created 
a comprehensive spreadsheet of assessment performance levels and proficiencies, as well as 
a school composition report. In short, the state remained responsive to the data needs and 
comfort levels of data users. 


When requests for information go beyond what is regularly available, an initial data request 
process with a brief form allows state data staff to pull the data for the requestor, if possible and 
in compliance with standard privacy protections. The form clarifies the request and allows the 
data team to keep track of requests to inform planning for additional public reporting. 
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Beyond this level of information is a tier for researchers who want access to restricted-use or 
suppressed data. In these cases, researchers must submit a more official and detailed proposal 
application. The data team reviews the request to determine if appropriate suppression is 
possible and to what level the research may benefit the SEA. The latter is always positive, but 
well-designed studies can be approved even if they do not directly benefit the SEA. 


Data Ethics 


A third area in which the West Virginia Department of Education has been meaningfully 
intentional is in communicating with data collectors about ethics. The state emphasizes that 
data collectors must remember that data represent students, noting that when local staff 

focus on accountability, they can forget that each number represents a real student. The SEA 
encourages local staff to think of the data in terms of what they demonstrate about students 
and the struggles they may be facing. For example, the importance of attendance data is not 
just that they are compulsory, but that they offer a chance to investigate whether students have 
appropriate transportation, whether they have unreported illnesses, or whether they are facing 
unknown struggles at home. The SEA reminds local staff to see the children behind the charts 
and remember that numbers only tell part of the story. 


The SEA has found that it often is reassuring to local staff to see that the SEA is viewing the 
data in this manner. Because the SEA’s official role is monitoring and compliance, LEAs may 
feel pressure regarding reporting. Knowing that the SEA goes beyond merely caring about 
compliance to thinking about what the data can tell educators about individual students 
and how those data can be used to assist students’ needs can help to build connections and 
collaboration between the different agency levels. 


Loudoun County Public Schools (VA): Focusing on Interoperability and 
Increasing Transparency 


For Loudoun County Public Schools (VA), the integration of data governance and data strategy has 
guided the district to a focus on data analytics and data science. As district data leaders recognized 
the need for more centralized and strategic data governance, they worked to democratize their 
approach to governance by empowering team members and increasing data transparency. 


A Need for Interoperability 


Loudoun County Public Schools (VA) initially began its interoperability efforts by developing 
elements, creating definitions, and working on mapping both strategic and tactical plans and 
goals. Over time, the team identified key gaps in knowledge and resources, and team members 
realized that they needed to accelerate their interoperability work and secure additional funding 
to support their interoperability goals. After considering their options, the team partnered 

with an outside organization to enhance its interoperability work. Financially, this work was 
supported by a grant to create data analytics that provided unified student assessment data. 


These arrangements and tools provided the acceleration of interoperability that the district 
needed. Shared code and connectors clarified relationships within the data and allowed the 
data team to handle technical issues team members had encountered with the use of different 
systems. A data import tool made data management much easier, allowing the team to quickly 
integrate data. 


The grant funding supported the team’s effort to develop a unified student assessment 
application that provides the basis for data dashboards for teachers. These dashboards offer 
quick, useful indicators, as well as data visualizations for teachers that connect and demonstrate 
patterns in the data from all student assessments. 


Forum Guide to Strategies for Education Data Collection and Reporting 27 


Before these efforts, the team had been focused on the storming stage of team development,” 
unable to agree upon definitions and uses of elements. Members now had access to different 
use cases for elements, allowing teams to move forward from these types of inconsistencies. 
Therefore the team reduced time spent on data literacy because users are given a clear 
framework that allows them to see where their needs fit. 


Increasing Data Transparency for Stakeholders 


Data leaders worked with different educational units to create an effective data governance 
team. The team includes individuals identified by their departments as having data science 
skills, and senior leadership grants them the authority to speak on their behalf. The team meets 
biweekly to move the district’s data analytics forward. 


The team also has worked to mitigate the concerns expressed by different departments as 

they increase the focus on data science. For example, some staff members were concerned 
about sharing human resources data. Concerns ranged from worries about how data might be 
perceived by stakeholders to fears that limited time and resources could not be spent on data 
sharing. However, senior leadership in the district has supported the perspective that there 
must be data transparency, and the greatest concern should be data accuracy. Also, the district 
will provide necessary clarification or disclaimers for data that may be confusing to the average 
stakeholder: for example, explaining that reports may reflect data collected at different times, or 
that terms may be defined and used differently at different levels or across educational units. 


Ultimately, the data team—and district leaders—understand the key role transparency plays 

in the district’s data strategy. Integrating transparency means that difficult questions may be 
raised, but these questions are part of why the data are made available to the public. The data 
belong to the community and its stakeholders, and they need to be able to understand what is 
happening in the district. Therefore, the district’s data strategy uses its focus on data analytics 
to improve transparency. 


Moving Forward 


As district staff have improved the interoperability of the LEA’s data and increased the use of 
data analytics, they have worked with teachers to ensure that they understand the data and 
solicit their feedback about ways to improve data dashboards and visualizations. A group of 
teachers has reviewed the visualizations, allowing the data team to realize that some needed to 
be presented more clearly and simply. Moving to analytics and visualizations also has allowed 
the team to work with teachers to increase their data literacy. A data app for teachers provides 
five guiding questions, giving them prompts and allowing a new understanding of the data. 
Making the process inquiry-based has allowed teachers to understand what the data mean 
within the context of instructional questions. They now are achieving greater levels of data 
literacy by working with their student’s actual data. 


Beyond this direct work with teachers, the data team has plans to get students involved in data 
science, giving them access to real data. In the coming years, they hope to include data science 
in the district’s curriculum. 


13 For more information about the storming stage, see 5 Stages of Team Development: Tuckman's Group Development, 
from https://project-management.com/stages-of-team-development/ 
14 For more information on data governance in Loudoun County Public Schools (VA), see the case study that 


begins on page 44 of the Forum Guide to Data Governance, available at https://nces.ed.gov/forum/pub_2020083.asp. 
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Data Destruction Data Privacy and Security 


Loudoun County Public Schools (VA) requires any Loudoun County Public Schools (VA) received the 
parties using its data to follow the Virginia Data designation of Consortium for School Networking 
Protection Agreement (https://www.lcps.org/cms/ (CoSN) Trusted Learning Environment (TLE; https:// 
lib/VA01000195/Centricity/Domain/111/20 Virginia trustedlearning.org/). This designation requires a 
School Data Privacy Agreement DPA FINAL 7-25-19. rigorous certification process, and signals that a 

pdf), building this agreement into the contracting district has taken strong and measurable steps to help 
process. Users must adhere to or exceed the data ensure the privacy of student data. The district has 
destruction component. Though the district is not built the TLE elements into its policies. Additionally, 
able to actively validate adherence at this point, they data leaders have developed a course for teachers on 
address the issue through the non-renewal process. data privacy, which they take each year. 


Wisconsin Department of Public Instruction (DPI): Strong Data Quality Measures and 
Agile Leadership Transform Strategic Data Use 


The Wisconsin Information System for Education (WISE) comprises multiple interoperable tools 
that support data collection to meet all state and federal reporting requirements. The complexity 
of these interoperable systems drove state data leaders to establish formalized data and project 
governance, as well as a structured data quality process. While the state’s foundational priority 
is collecting and sharing required data, a specific focus on data quality and transparency has 
allowed Wisconsin to be more strategic in its coordination, analysis, and use of data. 


Agile Leadership and the Scrum Process 


From a structural and process perspective, the Wisconsin Department of Public Instruction's 
(DPI's) data strategy focuses on one major project management philosophy: agile development 
practices that use the scrum process at the team level, which then is scaled. The agile 
development methodology is an iterative approach in which large projects are broken down 
into more manageable tasks tackled in short iterations or “sprints,” empowered by small teams. 
The scrum team framework is a team design with specific roles and teamwork expectations, in 
which the members work together to deliver required product increments. Wisconsin uses these 
concepts in tandem to direct its product development and data strategy. 


DPI's product development revolves around an agile leadership mindset. The philosophy focuses 
on satisfying the customer (in this case, program areas or LEAs) through early and continuous 
delivery of valuable software and data solutions. In the agile approach, team members identify 
what they are working toward with the customer, and the team begins by building a small 
initial piece to get feedback from the customer. The team continues to develop new iterations, 
rolling out small pieces every 2 weeks (the time of the agency’s “sprint” cycle). The belief is that 
constant feedback allows for a better product, as the teams interact regularly with customers 
throughout the sprints. The state’s data leaders find that they have been able to connect better 
with customers, built a relationship based on trust, and have bridged gaps between program 
areas and IT. Advisory groups consisting of LEA users for specific products were established 

to receive continuous feedback on product developments, which ensures development is 
prioritized based on the most important needs. 


Within the agile mindset, Wisconsin also depends on the scrum team, a structure that encourages 
high levels of communication among team members and an integrated working environment. 
Each scrum team is empowered to deliver solutions based on an assigned vision, and each has 
standard team roles. The product owner’s main responsibility is to answer the question, “What 

is the team doing next?” This person prioritizes key tasks and is responsible for coordinating the 
product vision and conveying it to the development team. The scrum master is considered the 
process owner. This person helps remove impediments, facilitates meetings, and works with the 
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product owner to make sure the backlog is in good shape. Finally, the development team consists 
of the business analyst, the quality assurance analyst, and the developers. Depending on the 
scrum team, the development team may range from three to seven members. 


Before implementing the scrum process as a core element of project and data governance, 
the state conducted development efforts that were not as streamlined, leading to potential 
redundancies or unidentified needs. Additionally, these projects used traditional waterfall 
project management methods, which map out a project into distinct, sequential phases, with 
each new phase beginning only when the prior phase has been completed. Over time, data 
leaders have made changes to the entire process to increase productivity, collaboration, and 
transparency. They now have the timely and accurate data they need to identify and provide 
needed resources, support students and educators, and continually improve processes. 


Wisconsin now uses a scaling framework for its approach to project governance. It comprises 
multiple scrum teams, the WISE Leadership Team, the WISE Steering Committee, and the IT 
Project Request and Prioritization Process. 


e¢ Scrum teams (application development, data warehouse, and DevOps) use the scaling 
framework as an agile development methodology, which uses a strategy that allows 
solutions to be delivered in usable and workable iterations. Each program area, or core 
product, has an assigned scrum team. Each scrum team has one individual assigned to 
the role of product owner, a scrum master, and one or more team members assigned to 
the development team (analysts, developers, and quality assurance). 

e The WISE Leadership Team, which meets weekly, is made up of the IT management 
team, scrum team product owners, and other key team members. This team handles 
the project request process, which involves a weekly review of any project requests 
entered by agency staff through a form on the agency’s intranet site. The team 
determines whether the project request can be assigned directly to a scrum team or if it 
needs review and prioritization by the WISE Steering Committee. The leadership team 
also communicates across the agency about items that may affect more than one team. 

e Although the WISE Steering Committee originally was developed for the WISEdata 
project, the committee now is a cross-agency group that covers the entire WISE 
product suite. It includes IT directors and program area directors from any program 
area that has data at the DPI, essentially every division and team in the agency. The 
steering committee prioritizes project work using a decision protocol it developed 
itself, which is crucial when program areas are competing for scrum team or staff time. 
The committee informs the product roadmap, following the group’s central goals of 
transparency and criterion-driven, consensus-based decisionmaking. The steering 
committee also represents the policy tier of the data governance structure at DPI and 
can make decisions and set priorities on that level. 


The scaling framework allows DPI to coordinate and facilitate work between multiple scrum 
teams and also to provide accountability and transparency. Each scrum team performs a daily 
scrum stand-up. This meeting lasts 15 minutes or less, and all team members share information 
based on three questions: What did you do yesterday? What are you doing today? What is 
standing in your way? 


Like the daily scrum standups, there also is a daily scrum of scrums, or scaled daily scrum 
meeting. This meeting consists of one representative from each scrum team. The purpose of 
the meeting is to discuss how teams can work together efficiently, provide team updates, and 
identify and resolve any dependencies between teams. 


30 Forum Guide to Strategies for Education Data Collection and Reporting 


Project Roadmaps 


Another key element of DPI’s data strategy is the use of product roadmaps, which are used to 
define targeted deliverable goals, communicate plans to stakeholders, and help keep teams 

on track from a high-level perspective. Each project—that is, any new request for data, an 
application, or a new dashboard, graph, report, map, or visualization— is discussed by the 
leadership committee and assigned to a particular scrum team, unless there is a need to discuss 
the project at a higher level. The project then is added to a product roadmap associated with 
the assigned scrum team. If a project is larger or may impact multiple teams, it is brought to the 
WISE Steering Committee for discussion before being approved to be added to a roadmap and 
moved forward. 


The WISE Leadership Team meets three times a year to review the roadmaps from all teams to 
make sure they align with agency priorities and goals. The team then presents these roadmaps 
to the WISE Steering Committee. Sharing this information lets stakeholders in program areas or 
LEAs understand the status and timing of their projects, as well as how they fit into the larger 
goals of DPI. This helps agency leadership communicate the reasoning behind certain decisions 
and allows stakeholders a better understanding of scheduling and priorities. For example, 

at one point requests came in at the same time to integrate both financial data and career 

and technical education (CTE) data into the data system. As these were large-scale tasks that 
could not be accomplished simultaneously, the discussion was brought to the WISE Steering 
Committee (which includes the program area directors). When the steering committee reviewed 
the roadmap, using the decisionmaking criteria it had developed itself, it was agreed that 
prioritizing the CTE data integration was the best choice. 


Strong Relationships Support Data Quality 


One of the ways DPI maintains its high-quality data is through strong relationships between the 
IT team and the program areas. When data are submitted to the state’s data warehouse, the IT 
team works with program areas to determine how the information can best be visualized and 
provided to stakeholders. IT also works through a data quality review process with the program 
areas before the publishing of any reports. This process uses warnings and error checks within 
the system to allow both sides to be aware of any discrepancies. In this way, data quality 
becomes a shared responsibility owned by the program area and IT. The IT team is working 
continually to reduce redundancies and make the system work better for LEAs. 


These relationships and mutual understanding of needs allow both groups to work successfully 
with WISEdata, the state’s multi-vendor, open data collection system that leverages an 
application programming interface (API) toolset. WISEdata allows school districts, charter 
schools, and private schools participating in a parental choice program to submit data to DPI 
from the SIS vendor of their choice. Under a plan approved by the legislature, DPI created the 
system to achieve multiple goals: to meet all required state and federal reporting mandates; 
present data through the secure WISEdash data portal to support continuous improvement 
planning, data quality, and early warning; eliminate duplicate data collection tools and 
processes; and partner with SIS vendors on data collection standards to make high-quality data 
available more easily and frequently. 


WISEdash: Providing Data to Stakeholders and Informing Early Warning Systems 


Data are available via WISEdash on two separate portals that use the same underlying software: 
a public portal, open to all users, and a secure portal, available to districts and schools to view 
their data via a secure, role-based login. DPI shares summarized and redacted data (to protect 
student privacy) with public stakeholders via the WISEdash Public Portal, a data portal that 
uses dashboards to provide multi-year education data about Wisconsin schools. Data on the 
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portal are driven mostly by required reporting: for example, enrollment data or achievement 
data for various student groups. Data are available by school, district, or aggregated at the 
state level; can be displayed for multiple years; and can be grouped and filtered by a variety 
of demographics, including grade level, gender, race/ethnicity, economic status, disability, 
English proficiency, and migrant status. Statewide data download files also are available. Data 
leaders worked with different stakeholders, such as state legislators, parents, and reporters, 
to determine which types of data and data visualizations would be most useful. The most 
requested feature was a comparison tool for schools and LEAs. As a public reporting tool, 
WISEdash is used by districts, schools, parents, researchers, media, and other community 
members to view data published by DPI. 


WISEdash for Districts, the secure portal, has a carefully designed system of role-based security, 
allowing different staff varying levels of data access. Some only see summary data, while others 
can see student profiles and other more sensitive information. WISEdash for Districts provides 
multiple tools, including supports for data inquiry and continuous improvement planning; data 
and dashboards for district-wide use of student data to drive school improvement; dashboards 
to help with district and school data verification and comparisons, such as certified data from 
one year to the next, for upcoming snapshots; and a secure platform to protect student privacy 
while viewing student outcomes. Much of what is housed on the secure portal is driven by 

user requests for items such as new data, dashboards, or visualizations. These requests are 
sometimes internal (from the SEA program area teams), but often are from LEAs throughout the 
state. Wisconsin has many small LEAs that have benefitted from having their data consolidated 
in one data dashboard system for them to use for analysis instead of having to build one. 


Because there are such robust data in WISEdash, the state has been able to use it to develop 
varied early warning systems. An early warning system provides information to help schools 
identify students who are not on track for desirable outcomes. The Dropout Early Warning 
System (DEWS) and the College and Career Readiness Early Warning System (CCREWS) are 
available in WISEdash, as well as Chronic Absenteeism and Free Application for Financial 
Student Aid (FAFSA) Filing Status. Therefore, instead of needing to create their early warning 
system (EWS), LEAs can use the statewide EWS options available through WISEdash.* 


In the Future 


DPI’s dual focus on data quality and More information about project management can be 
collaborative project governance has found in the Project Management Institute’s A Guide to 
allowed the state to streamline and improve _| the Project Management Body of Knowledge (PMBOK® 
its data strategy processes in recent years. Guide): https://www.pmi.org/pmbok-guide-standards 


Data leaders suggest that a potential next 
step is to work on adding additional supports in the WISE tools for educators. The agency has 
adopted the same toolset, which resides in WISEdash, for continuous improvement planning for 
all schools and districts identified within the accountability system, providing a common and 
robust method for conducting such planning. While the state recognizes that district and school 
administrative staff use WISEdash and other secure tools for continuous improvement planning, 
data quality, and student support, it plans to also help educators navigate through and use the 
tools for student and classroom support. The early warning indicators currently in use are a 
piece of that, but agency leadership seeks to expand the support available to those working 
directly with DPI’s students via local benchmark and classroom assessment data availability and 
the integration of other data sources. 


15 For more information on the Wisconsin DPI’s early warning systems, see the case study that begins on page 48 of 
the Forum Guide to Early Warning Systems, available at https://nces.ed.gov/forum/pub_2019035.asp. 
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Building a Culture that Values Data and Promotes Public Use 


Identify Data Needs to Answer Key Agency Questions: Use the learning agenda 
process to identify and prioritize the agency's key questions and the data needed to 
answer them. 

Assess and Balance the Needs of Stakeholders: Identify and engage stakeholders 
throughout the data lifecycle to identify stakeholder needs and to incorporate 
stakeholder feedback into government priorities to maximize entrepreneurship, 
innovation, scientific discovery, economic growth, and the public good. 

Champion Data Use: Leaders set an example, incorporating data in decisionmaking 
and targeting resources to maximize the value of data for decisionmaking, 
accountability, and the public good. 

Use Data to Guide Decisionmaking: Effectively, routinely, transparently, and 
appropriately use data in policy, planning, and operations to guide decisionmaking; 
share the data and analyses behind those decisions. 

Prepare to Share: Assess and proactively address the procedural, regulatory, legal, 
and cultural barriers to sharing data within and across federal agencies, as well as with 
external partners. 

Convey Insights from Data: Use a range of communication tools and techniques to 
effectively present insights from data to a broad set of audiences. 

Increase Accountability of Federal Spending: Align federal spending data with 
performance data to enable the public to understand the results of federal investments 
and to support informed decisionmaking. 

Monitor and Address Public Perceptions: Regularly assess and address public 
confidence in the value, accuracy, objectivity, and privacy protection of federal data to 
make strategic improvements, advance agency missions, and improve public messages 
about planned and potential uses of federal data. 

Connect Data Functions Across Agencies: Establish Communities of Practice for 
common agency data functions (for example, data management, access, analytics, 
informatics, user support) to promote efficiency, collaboration, and coordination. 
Provide Resources to Explicitly Leverage Data Assets: Ensure that sufficient human 
and fiscal resources are available to support using data for agency decisionmaking and 
accountability and to spur commercialization, innovation, and public use. 
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Governing, Managing, and Protecting Data 
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Prioritize Data Governance: Ensure there are sufficient authorities, roles, 
organizational structures, policies, and resources in place to transparently support the 
Management, maintenance, and use of strategic data assets. 

Govern Data to Protect Confidentiality and Privacy: Ensure there are sufficient 
authorities, roles, organizational structures, policies, and resources in place to provide 
appropriate access to confidential data and maintain public trust and safeguard privacy. 
Protect Data Integrity: Emphasize state-of-the-art data security in information 
technology (IT) security practices for every system that is refreshed, architected, or 
replaced to address current and emerging threats; foster innovation and leverage new 
technologies to maintain protection. 

Convey Data Authenticity: Disseminate data sets such that their authenticity is 
discoverable and verifiable by users throughout the information lifecycle, consistent 
with open data practice, and encourage appropriate attribution from users. 

Assess Maturity: Evaluate the maturity of all aspects of agency data capabilities to 
inform priorities for strategic resource investment. 

Inventory Data Assets: Maintain an inventory of data assets with sufficient 
completeness, quality, and metadata to facilitate discovery and collaboration in support 
of informing key agency questions and meeting stakeholder needs. 

Recognize the Value of Data Assets: Assign a value to data assets based on maturity, 
key agency questions, stakeholder feedback, and applicable law and regulation to 
appropriately prioritize and document resource decisions. 

Manage with a Long View: Include data investments in annual capital planning 
processes and associated guidance to ensure appropriated funds are being used 
efficiently to leverage data as a strategic long-term asset. 

Maintain Data Documentation: Store up-to-date and comprehensive data 
documentation in accessible repositories to facilitate use and document quality, 
utility, and provenance in support of informing key agency questions and meeting 
stakeholder needs. 

Leverage Data Standards: Adopt or adapt, create if needed, and implement data 
standards within relevant communities of interest to maximize data quality and 
facilitate use, access, sharing, and interoperability. 

Align Agreements with Data Management Requirements: Establish terms and 
conditions for contracts, grants, cooperative agreements, and other agreements that 
meet data management requirements for processing, storage, access, transmission, 
and disposition. 

Identify Opportunities to Overcome Resource Obstacles: Coordinate with 
stakeholders to identify mutually-acceptable cost recovery, shared service, or 
partnership opportunities to enable data access while conserving available resources 
to meet user demand. 

Allow Amendment: Establish clear procedures to allow members of the public to 
access and amend federal data about themselves, as appropriate and in accordance 
with federal laws, regulations, and policies, to safeguard privacy, reduce potential harm 
from inaccurate data, and promote transparency. 

Enhance Data Preservation: Preserve federal data in accordance with applicable law, 
regulation, policy, approved schedules, and mission relevance. 
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Coordinate Federal Data Assets: Coordinate and share data assets across federal 
agencies to advance progress on shared and similar objectives, fulfill broader federal 
information needs, and reduce collection burden. 

Share Data Between State, Local, and Tribal Governments and Federal Agencies: 
Facilitate data sharing between state, local, and tribal governments and the federal 
government, where relevant and appropriate and with proper protections, particularly 
for programs that are federally funded and locally administered, to enable richer 
analyses for more informed decisionmaking. 


Promoting Efficient and Appropriate Data Use 


Increase Capacity for Data Management and Analysis: Educate and empower the 
federal workforce by investing in training, tools, communities, and other opportunities 
to expand capacity for critical data-related activities such as analysis and evaluation, 
data management, and privacy protection. 

Align Quality with Intended Use: Data likely to inform important public policy or 
private sector decisions must be of appropriate utility, integrity, and objectivity. 
Design Data for Use and Re-use: Design new data collections with the end use and 
users in mind to ensure that data are necessary and of high enough quality to meet 
planned and future agency and stakeholder needs. 

Communicate Planned and Potential Uses of Data: Review data collection 
procedures to update and improve how planned and future uses of data are 
communicated, promoting public trust through transparency. 

Explicitly Communicate Allowable Use: Regularly employ descriptive metadata that 
provides clarity about access and use restrictions for federal data, explicitly recognizes 
and safeguards applicable intellectual property rights, conveys attribution as needed, 
and optimizes potential value to stakeholders to maximize appropriate legal use. 
Harness Safe Data Linkage: Test, review, and deploy data linkage and analysis tools 
that use secure and privacy-protective technologies to address key agency questions 
and meet stakeholder needs while protecting privacy. 

Promote Wide Access: Promote equitable and appropriate access to data in open, 
machine-readable form and through multiple mechanisms, including through both 
federal and non-federal providers, to meet stakeholder needs while protecting privacy, 
confidentiality, and proprietary interests. 

Diversify Data Access Methods: Invest in the creation and usability of multiple tiers 
of access to make data as accessible as possible while minimizing privacy risk and 
protecting confidentiality. 

Review Data Releases for Disclosure Risk: Review federal data releases to the public 
to assess and minimize the risk of re-identification, consistent with applicable laws and 
policies, and publish reviews to promote transparency and public trust. 

Leverage Partnerships: Create and sustain partnerships that facilitate innovation with 
commercial, academic, and other partners to advance agency mission and maximize 
economic opportunities, intellectual value, and the public good. 

Leverage Buying Power: Monitor needs and systematically leverage buying power for 
private-sector data assets, services, and infrastructure to promote efficiency and reduce 
federal costs. 
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Leverage Collaborative Computing Platforms: Periodically review and optimize 

the use of modern collaborative computing platforms to minimize costs, improve 
performance, and increase use. 

Support Federal Stakeholders: Engage with relevant agencies to share expert knowledge 
of data assets, promote wider use, improve usability and quality, and meet mission goals. 
Support Non-Federal Stakeholders: Engage with industry, academic, and other 
non-federal users of data to share expert knowledge of data assets, promote wider use, 
improve usability and quality, and advance innovation and commercialization. 
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Appendix B: Resources for Strategies for 
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Legal References 
Americans With Disabilities Act of 1990, Pub. L. No. 101-336, 104 Stat. 328 (1990). 
https://www.ada.gov/pubs/adastatute08.htm 


Family Educational Rights and Privacy Act, 20 U.S.C. § 1232g (1974). 
https://www2.ed.gov/policy/gen/guid/fpco/ferpa/ 


Additional Resources 


California Department of Education Guidance for Changing a Student's Gender in the 
California Longitudinal Pupil Achievement Data System: https://www.cde.ca.gov/ds/sp/cl/ 
calpadsupdflashi58.asp 


Common Education Data Standards (CEDS): https://ceds.ed.gov/ 


Current Measures of Sexual Orientation and Gender Identity in Federal Surveys: https://nces. 
ed.gov/FCSM/pdf/buda5.pdf 


EDFacts Disclosure Review Board: https://www2.ed.gov/about/inits/ed/edfacts/ed-disclosure- 
avoidance-overview.pdf 


Elementary and Secondary Information System (EISi): https://nces.ed.gov/ccd/elsi 


Fairfax County Public Schools (VA) Student Information System (SIS): https://www.fcps.edu/ 
resources/technology/student-information-system-sis-fcps 


Five Stages of Team Development: Tuckman's Group Development: https://project-management. 
com/stages-of-team-development/ 


Florida Association for Testing Administrators: https://www.floridatestadmin.com/ 


Kentucky Student Information System Data Standards: https://education.ky.gov/districts/tech/ 
sis/Pages/KSIS-Data-Standards.aspx 


National Center for Education Statistics (NCES) Confidentiality Procedures: https://nces.ed.gov/ 
statprog/confproc.asp 


Privacy Technical Assistance Center (PTAC): https://studentprivacy.ed.gov 


PTAC Best Practices for Data Destruction: https://studentprivacy.ed.gov/resources/best- 
practices-data-destruction 


PTAC Data Governance Checklist: https://nces.ed.gov/Forum/pdf/data governance checklist.pdf 


Project Management Institute’s A Guide to the Project Management Body of Knowledge (PMBOK® 
Guide): https://www.pmi.org/pmbok-guide-standards 
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Statewide Longitudinal Data System (SLDS) Data Governance Toolkit: https://slds.grads360. 
org/#program/data-governance 


SLDS Data Maturity Model: https://slds.ed.gov/#communities/pdc/documents/19350 
SLDS Data Use Strategy: https://nces.ed.gov/programs/slds/pdf/data_use_strategy.pdf 


SLDS Technical Brief: Statistical Methods for Protecting Personally Identifiable Information in 
Aggregate Reporting: https://nces.ed.gov/pubs2011/2011603.pdf 


Trusted Learning Environment Seal Program: https://trustedlearning.org/ 


Virginia School Data Policy Agreement: https://www.|cps.org/cms/lib/VA01000195/Centricity/ 
Domain/111/20 Virginia School Data Privacy Agreement DPA FINAL 7-25-19.pdf 


Washington State Office of Superintendent of Public Instruction's K-12 Data Governance 
workgroup: https://www.k12.wa.us/about-ospi/workgroups-committees/currently-meeting- 
workgroups/k-12-data-governance 


Washington State Office of Superintendent of Public Instruction K-12 Data Governance: https:// 
www.k12.wa.us/sites/default/files/public/cisl/pubdocs/DataGovernanceManual.pdf 


West Virginia Student Data Accessibility, Transparency and Accountability Act: http://www. 
wvlegislature.gov/WVCODE/ChapterEntire.cfm?chap=18&art=2&section=5H#02 


National Forum on Education Statistics Resources 


Forum Guide to Building a Culture of Quality Data: A School and District Resource (2004) 
https://nces.ed.gov/forum/pub 2005801.asp 

This guide was developed by the Forum’s Data Quality Task Force to help schools and school 
districts improve the quality of data they collect and to provide processes for developing a 
“Culture of Quality Data” by focusing on data entry—getting things right at the source. 


Forum Guide to Data Ethics (2010) 
http://nces.ed.gov/forum/pub 2010801.asp 


While laws set the legal parameters that govern data use, ethics establish fundamental principles 
of “right and wrong” that are critical to the appropriate management and use of education 

data in the technology age. This guide reflects the experience and judgment of seasoned data 
managers; while there is no mandate to follow these principles, it is hoped that the contents will 
prove a useful reference to others in their work. 


Forum Guide to Data Ethics Online Course (2010) 
https://nces.ed.gov/forum/dataethics course.asp 

This course is based on The Forum Guide to Data Ethics and is focused on how ethical 
principles apply to education data. The course is intended for any person who handles data in 
an education organization. 


Forum Guide to Data Governance (2020) 

https://nces.ed.gov/forum/pub_2020083.asp 

This resource provides timely and useful best practices, examples, and resources for agencies 
implementing or updating their data governance programs. It provides an overview of data 
governance; discusses effective data governance practices, structures, and essential elements; 
describes how to meet privacy and security requirements while also meeting data accessibility 
and sharing needs; and includes detailed case studies from education agencies in their data 
governance efforts. 
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Forum Guide to Data Visualization: A Resource for Education Agencies (2016) 
https://nces.ed.gov/forum/pub_2017016.asp 

This resource recommends data visualization practices that will help education agencies 
communicate data meaning in visual formats that are accessible, accurate, and actionable for a 
wide range of education stakeholders. Although this resource is designed for staff in education 
agencies, many of the visualization principles apply to other fields as well. 


Forum Guide to Early Warning Systems (2018) 
https://nces.ed.gov/forum/pub_2019035.asp 

This resource provides information and best practices that will help education agencies plan, 
develop,implement, and use an early warning system in their agency to inform interventions 
that improve student outcomes. This document focuses on early warning systems and their data 
from the perspective of the education data community. 


Forum Guide to Education Data Privacy (2016) 
https://nces.ed.gov/forum/pub 2016096.asp 


This resource provides SEAs and LEAs with best practice information to use in assisting 
school staff in protecting the confidentiality of student data in instructional and administrative 
practices. SEAs and LEAs may also find the guide useful in developing privacy programs and 
related professional development programs. 


Forum Guide to Metadata: The Meaning Behind Education Data (2009) 
https://nces.ed.gov/forum/pub_2009805.asp 

This resource offers best practice concepts, definitions, implementation strategies, and 
templates/tools for an audience of data, technology, and program staff in SEAs and LEAs. It is 
hoped that this resource will improve this audience’s awareness and understanding of metadata 
and, subsequently, the quality of the data in the systems they maintain. 


Forum Guide to Supporting Data Access for Researchers: A Local Education Agency 
Perspective (2014) 

https://nces.ed.gov/forum/pub_2014801.asp 

This resource recommends a set of core practices, operations, and templates that can be 
adopted and adapted by LEAs as they consider how to respond to requests for both new and 
existing data about the education enterprise. 


Forum Guide to Supporting Data Access for Researchers: A State Education Agency 
Perspective (2012) 

https://nces.ed.gov/forum/pub_2012809.asp 

This resource recommends a set of core practices, operations, and templates that can be 
adopted and adapted by SEAs as they consider how to respond to requests for data about the 
education enterprise, including data maintained in longitudinal data systems. 


Forum Guide to Taking Action with Education Data (2013) 
https://nces.ed.gov/forum/pub_2013801.asp 

This resource provides practical information about the knowledge, skills, and abilities needed to 
identify, access, interpret, and use data to improve instruction in classrooms and the operation 
of schools, LEAs, and SEAs. 
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Examples of Data Strategy in the U.S. Federal Government 


U.S. Department of Commerce, National Oceanic and Atmospheric Administration 
(NOAA) - NOAA Data Strategy: Maximizing the Value of NOAA Data 
https://nrc.noaa.gov/Portals/O/Final%20Data%2OStrategy.pdf 

The purpose of the NOAA Data Strategy is to dramatically accelerate the use of data across the 
agency and with other key partners, maximize openness and transparency, deliver on mission, 
and steward resources while protecting quality, integrity, security, privacy, and confidentiality. 
The overall strategy is designed to serve as a framework for consistency that builds upon 
existing laws and regulations related to how NOAA uses and manages data while being flexible 
and adaptable to external influences such as new policies, Executive Orders, stakeholder input, 
and new technologies that drive innovation within the agency. 


U.S. Department of Defense (DoD) - DoD Data Strategy 
https://media.defense.gov/2020/Oct/08/2002514180/-1/-1/0/DOD-DATA-STRATEGY.PDF 
The DoD Data Strategy, as a key component of the Department’s Digital Modernization program, 
supports the National Defense Strategy (NDS) by enhancing military effectiveness through 
access to accurate, timely, and secure data. In addition to combat effectiveness, DoD leaders— 
including members of the Office of the Secretary of Defense (OSD), the Military Departments, 
the Office of the Chairman of the Joint Chiefs of Staff (CJCS) and the Joint Staff, Combatant 
Commands, Defense Agencies, and DoD Field Activities (referred to collectively in this strategy 
as Components)—require data-driven insights that provide a fair and accurate Department-wide 
representation of DoD operations and management. 


U.S. Department of Education — Data Strategy 
https://www.ed.gov/sites/default/files/cdo/ed-data-strategy.pdf 

The Department Data Strategy describes a vision, establishes strategic goals for advancing data 
capabilities, and envisions agency-wide outcomes. It establishes an ambitious vision as the 
point on the horizon: To realize the full potential of data to improve education outcomes and 
lead the nation in a new era of evidence-based policy insights and data-driven operations. This 
Department-wide effort will include discussions across the agency about data priorities that will 
help improve data maturity and the Department’s capabilities to leverage data, operationalize 
and optimize data governance, and drive cultural change for the benefit of internal and 
external stakeholders. 


U.S. Department of Health and Human Services (HHS) - 2018 HHS Data Strategy: 
Enhancing the HHS Evidence-Based Portfolio 
https://aspe.hhs.gov/pdf-report/2018-hhs-data-strategy-enhancing-hhs-evidence-based- 
portfolio 

The 2018 HHS Data Strategy focuses on improving the Department’s capacity to develop 
statistical evidence to support policymaking and program evaluation over the next six to eight 
years. As the principal internal advisory body to the Secretary of Health and Human Services 
on the Department’s data and statistical policy, the HHS Data Council develops, implements, 
and updates the Department’s data strategy. There are six priorities outlined in the strategy: 1) 
improving access to HHS data, 2) enhancing administrative data for research, 3) increasing data 
linkages across diverse data assets, 4) modernizing privacy protections, 5) increasing data policy 
coordination and information sharing across the department, and 6) building a 21st-century 
data-oriented workforce. 


U.S. Department of Justice (DOJ) - Data Strategy for the U.S. Department of Justice 
https://www. justice.gov/jmd/page/file/1135081/download 

The DOJ Data Strategy is a foundational framework that will enable the Department to build 
a standardized, programmatic approach to manage and share data as well as advance its data 
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communities. The long-term objective of the Data Strategy is to optimize the value of the 
Department’s data assets for use in its missions. Consistent with the Federal Data Strategy, 
amongst other federal statutory and regulatory requirements, the Data Strategy seeks to build 
enterprise capabilities for data management, information sharing, controlled access, and 
maintaining a modern and relevant data workforce. The long-term objective is to optimize the 
impact of information and related information technology (IT) investments on the mission and 
the people serving the mission. 


U.S. Federal Government - The Federal Data Strategy: Principles and Practices 
https://strategy.data.gov 

The mission of the Federal Data Strategy is to leverage the full value of federal data for mission, 
service, and the public good by guiding the federal government in practicing ethical governance, 
conscious design, and a learning culture. The Federal Data Strategy offers agencies guidance 
about managing and using federal data. It consists of principles and practices to deliver a more 
consistent approach to federal data stewardship, use, and access. The principles are a timeless, 
enduring framework for agencies. They are actionable, yet aspirational, goals for a 5- to 10-year 
time horizon, and the yearly Action Plans identify concrete steps for agencies to undertake to 
achieve this long-term vision. 


Examples of Data Strategy in Other State and Local Agencies 


Atlanta Public Schools (GA) - Data and Information Group 
https://www.atlantapublicschools.us/dig 

Comprising three departments - Research and Evaluation, Student Information Systems (SIS), 
and Testing and Assessment - the Data and Information Group (DIG) looks after the lifecycle 

of student data for Atlanta Public Schools (APS). DIG collects data; provides applications, 
protocols, and resources to track, protect and measure data; and through research, mines data 
for positive trends to help school and district leaders form better strategies for learning. Its 
purview is broad and the group works with teachers, parents, district staff, APS leaders, state 
agencies, and university research partners. 


Colorado Office of Information Technology - State of Colorado Data Strategy 
http://hermes.cde.state.co.us/drupal/islandora/object/co:11294/datastream/OBJ/view 
The State of Colorado Data Strategy was developed to ensure that state government 
policymakers and knowledge workers have the data and information they need to do their 
work. The State of Colorado Data Strategy outlines the state’s data management and governance 
program. It describes the key business drivers behind the program, the status of the as-is 
environment of data, the organizational alignment of the program, and the migration plan for 
moving forward with an enterprise approach. 


Connecticut Office of Policy and Management - Connecticut State Data Plan 
https://portal.ct.gov/CTData/Content/Connecticut-State-Data-Plan 

The Connecticut State Data Plan serves as a framework for the state’s executive branch agencies 
to engage in a consistent approach to data stewardship, use, and access. Its purpose is to 
connect the people and processes involved with data to promote communication between, 
and appropriate integration of, formerly siloed data, teams, and systems. The plan is organized 
around principles that represent a framework under which state agencies should organize 

and operate; focal points that represent areas where agencies should emphasize the sharing, 
integration, and availability of data; and goals that represent the desired outcomes of plan 
implementation. This framework is based on an approach currently being used at the federal 
level in the formulation of the Federal Data Strategy. 


Forum Guide to Strategies for Education Data Collection and Reporting 41 


Illinois State Board of Education - Department of Data Strategies and Analytics 
https://www.isbe.net/Pages/Data-Analysis.aspx 

The Department of Data Strategies and Analytics (DSA) coordinates annual reporting, collects 
data, implements data sharing, administers the data governance program, and analyzes data 
for policy and strategic planning related to Board goals and legislative requirements. DSA also 
advises agency staff on data governance policies and procedures. DSA assists in metadata 
collection, design, and implementation. DSA helps local education agencies in meeting their 
mission, vision, and goals. 


Public Schools of Brookline (MA) - Office of Strategy and Performance 
https://www.brookline.k12.ma.us/domain/721 

The Office of Strategy and Performance was created during the 2015-2016 school year to 
coordinate the district’s strategic planning, the district-wide management of student information 
systems, the use of student data, the district’s internal and external communications, and 

family and community outreach. The Office is a renamed, reorganized department comprised 
primarily of staff from what was previously called the Data Team. 


Tulsa Public Schools (OK) - Data Strategy and Analytics 
https://www.tulsaschools.org/about/teams/data-team 

The Data Strategy and Analytics Team supports teachers, schools, and other district team 
members by providing analytic reports, creating data tools, developing dashboards, fulfilling 
data requests, and conducting research and program evaluations. 
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